<img src='imgs/horse2zebra.gif' align="right" width=384>

<br><br><br>

CycleGAN

PyTorch | project page | paper

Torch implementation for learning an image-to-image translation (i.e. pix2pix) without input-output pairs, for example:

New: Please check out contrastive-unpaired-translation (CUT), our new unpaired image-to-image translation model that enables fast and memory-efficient training.

<img src="https://junyanz.github.io/CycleGAN/images/teaser_high_res.jpg" width="1000px"/>

Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
Jun-Yan Zhu*, Taesung Park*, Phillip Isola, Alexei A. Efros
Berkeley AI Research Lab, UC Berkeley
In ICCV 2017. (* equal contributions)

This package includes CycleGAN, pix2pix, as well as other methods like BiGAN/ALI and Apple's paper S+U learning.
The code was written by Jun-Yan Zhu and Taesung Park.
Update: Please check out PyTorch implementation for CycleGAN and pix2pix. The PyTorch version is under active development and can produce results comparable or better than this Torch version.

Other implementations:

<p><a href="https://github.com/leehomyc/cyclegan-1"> [Tensorflow]</a> (by Harry Yang), <a href="https://github.com/architrathore/CycleGAN/">[Tensorflow]</a> (by Archit Rathore), <a href="https://github.com/vanhuyz/CycleGAN-TensorFlow">[Tensorflow]</a> (by Van Huy), <a href="https://github.com/XHUJOY/CycleGAN-tensorflow">[Tensorflow]</a> (by Xiaowei Hu), <a href="https://github.com/LynnHo/CycleGAN-Tensorflow-Simple"> [Tensorflow-simple]</a> (by Zhenliang He), <a href="https://github.com/luoxier/CycleGAN_Tensorlayer"> [TensorLayer]</a> (by luoxier), <a href="https://github.com/Aixile/chainer-cyclegan">[Chainer]</a> (by Yanghua Jin), <a href="https://github.com/yunjey/mnist-svhn-transfer">[Minimal PyTorch]</a> (by yunjey), <a href="https://github.com/Ldpe2G/DeepLearningForFun/tree/master/Mxnet-Scala/CycleGAN">[Mxnet]</a> (by Ldpe2G), <a href="https://github.com/tjwei/GANotebooks">[lasagne/Keras]</a> (by tjwei), <a href="https://github.com/simontomaskarlsson/CycleGAN-Keras">[Keras]</a> (by Simon Karlsson)</p> </ul>

Applications

Monet Paintings to Photos

<img src="https://junyanz.github.io/CycleGAN/images/painting2photo.jpg" width="1000px"/>

Collection Style Transfer

<img src="https://junyanz.github.io/CycleGAN/images/photo2painting.jpg" width="1000px"/>

Object Transfiguration

<img src="https://junyanz.github.io/CycleGAN/images/objects.jpg" width="1000px"/>

Season Transfer

<img src="https://junyanz.github.io/CycleGAN/images/season.jpg" width="1000px"/>

Photo Enhancement: Narrow depth of field

<img src="https://junyanz.github.io/CycleGAN/images/photo_enhancement.jpg" width="1000px"/>

Prerequisites

• Linux or OSX
• NVIDIA GPU + CUDA CuDNN (CPU mode and CUDA without CuDNN may work with minimal modification, but untested)
• For MAC users, you need the Linux/GNU commands gfind and gwc, which can be installed with brew install findutils coreutils.

Getting Started

Installation

• Install torch and dependencies from https://github.com/torch/distro
• Install torch packages nngraph, class, display

luarocks install nngraph
luarocks install class
luarocks install https://raw.githubusercontent.com/szym/display/master/display-scm-0.rockspec

• Clone this repo:

git clone https://github.com/junyanz/CycleGAN
cd CycleGAN

Apply a Pre-trained Model

• Download the test photos (taken by Alexei Efros):

bash ./datasets/download_dataset.sh ae_photos

• Download the pre-trained model style_cezanne (For CPU model, use style_cezanne_cpu):

bash ./pretrained_models/download_model.sh style_cezanne

• Now, let's generate Paul Cézanne style images:

DATA_ROOT=./datasets/ae_photos name=style_cezanne_pretrained model=one_direction_test phase=test loadSize=256 fineSize=256 resize_or_crop="scale_width" th test.lua

The test results will be saved to ./results/style_cezanne_pretrained/latest_test/index.html.
Please refer to Model Zoo for more pre-trained models. ./examples/test_vangogh_style_on_ae_photos.sh is an example script that downloads the pretrained Van Gogh style network and runs it on Efros's photos.

Train

• Download a dataset (e.g. zebra and horse images from ImageNet):

bash ./datasets/download_dataset.sh horse2zebra

• Train a model:

DATA_ROOT=./datasets/horse2zebra name=horse2zebra_model th train.lua

• (CPU only) The same training command without using a GPU or CUDNN. Setting the environment variables gpu=0 cudnn=0 forces CPU only

DATA_ROOT=./datasets/horse2zebra name=horse2zebra_model gpu=0 cudnn=0 th train.lua

• (Optionally) start the display server to view results as the model trains. (See Display UI for more details):

th -ldisplay.start 8000 0.0.0.0

Test

• Finally, test the model:

DATA_ROOT=./datasets/horse2zebra name=horse2zebra_model phase=test th test.lua

The test results will be saved to an HTML file here: ./results/horse2zebra_model/latest_test/index.html.

Model Zoo

Download the pre-trained models with the following script. The model will be saved to ./checkpoints/model_name/latest_net_G.t7.

bash ./pretrained_models/download_model.sh model_name

• orange2apple (orange -> apple) and apple2orange: trained on ImageNet categories apple and orange.
• horse2zebra (horse -> zebra) and zebra2horse (zebra -> horse): trained on ImageNet categories horse and zebra.
• style_monet (landscape photo -> Monet painting style), style_vangogh (landscape photo -> Van Gogh painting style), style_ukiyoe (landscape photo -> Ukiyo-e painting style), style_cezanne (landscape photo -> Cezanne painting style): trained on paintings and Flickr landscape photos.
• monet2photo (Monet paintings -> real landscape): trained on paintings and Flickr landscape photographs.
• cityscapes_photo2label (street scene -> label) and cityscapes_label2photo (label -> street scene): trained on the Cityscapes dataset.
• map2sat (map -> aerial photo) and sat2map (aerial photo -> map): trained on Google maps.
• iphone2dslr_flower (iPhone photos of flowers -> DSLR photos of flowers): trained on Flickr photos.

CPU models can be downloaded using:

bash pretrained_models/download_model.sh <name>_cpu

, where <name> can be horse2zebra, style_monet, etc. You just need to append _cpu to the target model.

Training and Test Details

To train a model,

DATA_ROOT=/path/to/data/ name=expt_name th train.lua

Models are saved to ./checkpoints/expt_name (can be changed by passing checkpoint_dir=your_dir in train.lua).
See opt_train in options.lua for additional training options.

To test the model,

DATA_ROOT=/path/to/data/ name=expt_name phase=test th test.lua

This will run the model named expt_name in both directions on all images in /path/to/data/testA and /path/to/data/testB.
A webpage with result images will be saved to ./results/expt_name (can be changed by passing results_dir=your_dir in test.lua).
See opt_test in options.lua for additional test options. Please use model=one_direction_test if you only would like to generate outputs of the trained network in only one direction, and specify which_direction=AtoB or which_direction=BtoA to set the direction.

There are other options that can be used. For example, you can specify resize_or_crop=crop option to avoid resizing the image to squares. This is indeed how we trained GTA2Cityscapes model in the projet webpage and Cycada model. We prepared the images at 1024px resolution, and used resize_or_crop=crop fineSize=360 to work with the cropped images of size 360x360. We also used lambda_identity=1.0.

Datasets

Download the datasets using the following script. Many of the datasets were collected by other researchers. Please cite their papers if you use the data.

bash ./datasets/download_dataset.sh dataset_name

• facades: 400 images from the CMP Facades dataset. [Citation]
• cityscapes: 2975 images from the Cityscapes training set. [Citation]. Note: Due to license issue, we do not host the dataset on our repo. Please download the dataset directly from the Cityscapes webpage. Please refer to ./datasets/prepare_cityscapes_dataset.py for more detail.
• maps: 1096 training images scraped from Google Maps.
• horse2zebra: 939 horse images and 1177 zebra images downloaded from ImageNet using the keywords wild horse and zebra
• apple2orange: 996 apple images and 1020 orange images downloaded from ImageNet using the keywords apple and navel orange.
• summer2winter_yosemite: 1273 summer Yosemite images and 854 winter Yosemite images were downloaded using Flickr API. See more details in our paper.
• monet2photo, vangogh2photo, ukiyoe2photo, `ceza