SnapMix: Semantically Proportional Mixing for Augmenting Fine-grained Data (AAAI 2021)

PyTorch implementation of SnapMix | paper

Method Overview

Cite

@inproceedings{huang2021snapmix,
    title={SnapMix: Semantically Proportional Mixing for Augmenting Fine-grained Data},
    author={Shaoli Huang, Xinchao Wang, and Dacheng Tao},
    year={2021},
    booktitle={AAAI Conference on Artificial Intelligence},
}

Setup

Install Package Dependencies

torch
torchvision 
PyYAML
easydict
tqdm
scikit-learn
efficientnet_pytorch
pandas
opencv

Datasets

create a soft link to the dataset directory

CUB dataset

ln -s /your-path-to/CUB-dataset data/cub

Car dataset

ln -s /your-path-to/Car-dataset data/car

Aircraft dataset

ln -s /your-path-to/Aircraft-dataset data/aircraft

Training

Training with Imagenet pre-trained weights

1. Baseline and Baseline+

To train a model on CUB dataset using the Resnet-50 backbone,

python main.py # baseline

python main.py --midlevel # baseline+

To train model on other datasets using other network backbones, you can specify the following arguments:

--netname: name of network architectures (support 4 network families: ResNet,DenseNet,InceptionV3,EfficientNet)

--dataset: dataset name

For example,

python main.py --netname resnet18 --dataset cub # using the Resnet-18 backbone on CUB dataset

python main.py --netname efficientnet-b0 --dataset cub # using the EfficientNet-b0 backbone on CUB dataset

python main.py --netname inceptoinV3 --dataset aircraft # using the inceptionV3 backbone on Aircraft dataset

2. Training with mixing augmentation

Applying SnapMix in training ( we used the hyperparameter values (prob=1., beta=5) for SnapMix in most of the experiments.):

python main.py --mixmethod snapmix --beta 5 --netname resnet50 --dataset cub # baseline

python main.py --mixmethod snapmix --beta 5 --netname resnet50 --dataset cub --midlevel # baseline+

Applying other augmentation methods (currently support cutmix,cutout,and mixup) in training:

python main.py --mixmethod cutmix --beta 3 --netname resnet50 --dataset cub # training with CutMix

python main.py --mixmethod mixup --prob 0.5 --netname resnet50 --dataset cub # training with MixUp

3. Results

ResNet architecture.

| Backbone | Method | CUB | Car | Aircraft |
|:--------|:--------|--------:|------:|--------:| |Resnet-18 | Baseline| 82.35% | 91.15% | 87.80% |
|Resnet-18 | Baseline + SnapMix| 84.29% | 93.12% | 90.17% | |Resnet-34 | Baseline| 84.98% | 92.02% | 89.92% |
|Resnet-34 | Baseline + SnapMix| 87.06% | 93.95% | 92.36% | |Resnet-50 | Baseline| 85.49% | 93.04% | 91.07% |
|Resnet-50 | Baseline + SnapMix| 87.75% | 94.30% | 92.08% | |Resnet-101 | Baseline| 85.62% | 93.09% | 91.59% |
|Resnet-101 | Baseline + SnapMix| 88.45% | 94.44% | 93.74% | |Resnet-50 | Baseline+| 87.13% | 93.80% | 91.68% |
|Resnet-50 | Baseline+ + SnapMix| 88.70% | 95.00% | 93.24% | |Resnet-101 | Baseline+| 87.81% | 93.94% | 91.85% |
|Resnet-101 | Baseline+ + SnapMix| 89.32% | 94.84% | 94.05% |

InceptionV3 architecture.

| Backbone | Method | CUB | |:--------|:--------|--------:| |InceptionV3 | Baseline| 82.22% | |InceptionV3 | Baseline + SnapMix| 85.54%|

DenseNet architecture.

| Backbone | Method | CUB | |:--------|:--------|--------:| |DenseNet121 | Baseline| 84.23% |
|DenseNet121| Baseline + SnapMix| 87.42%|

Training from scratch

To train a model without using ImageNet pretrained weights:

python main.py --mixmethod snapmix --prob 0.5 --netname resnet18 --dataset cub --pretrained 0 # resnet-18 backbone

python main.py --mixmethod snapmix --prob 0.5 --netname resnet50 --dataset cub --pretrained 0 # resnet-50 backbone

2. Results

| Backbone | Method | CUB | |:--------|:--------|--------:| |Resnet-18 | Baseline| 64.98% | |Resnet-18 | Baseline + SnapMix| 70.31%| |Resnet-50 | Baseline| 66.92% |
|Resnet-50| Baseline + SnapMix| 72.17%|