Unbalanced Feature Transport for Exemplar-based Image Translation [CVPR 2021]

Update

• 04/2022: We include Marginal Contrastive Correspondence for Guided Image Generation [CVPR 2022] in this project. Marginal contrastive learning is introduced for building correspondence. Please check models/networks/nceloss.py and models/networks/correspondence.py.

Preparation

Clone the Synchronized-BatchNorm-PyTorch repository.

cd models/networks/
git clone https://github.com/vacancy/Synchronized-BatchNorm-PyTorch
cp -rf Synchronized-BatchNorm-PyTorch/sync_batchnorm .
cd ../../

VGG model for computing loss. Download from here, move it to models/.

For the preparation of datasets, please refer to CoCosNet.

Pretrained Models

The pretrained models can be downloaded from here. Saving the pretrained models in checkpoints.

Inference Using Pretrained Model

Then run the command

bash test_ade.sh

Training

Then run the command

bash train_ade.sh

If you want to use marginal contrastive learning for building corresondence, please add --mcl and set the weight --nce_w in train_xxx.sh. Then run:

bash train_xxx.sh

Citation

If you use this code for your research, please cite our papers.

@inproceedings{zhan2021unite,
  title={Unbalanced Feature Transport for Exemplar-based Image Translation},
  author={Zhan, Fangneng and Yu, Yingchen and Cui, Kaiwen and Zhang, Gongjie and Lu, Shijian and Pan, Jianxiong and Zhang, Changgong and Ma, Feiying and Xie, Xuansong and Miao, Chunyan},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={15028--15038},
  year={2021}
}

Acknowledgments

This code borrows heavily from CoCosNet. We also thank SPADE, Synchronized Normalization and Geometric Loss.