CainGAN

Pose Manipulation with Identity Preservation

Paper

Pose Manipulation with Identity Preservation. Andrei-Timotei Ardelean, Lucian Mircea Sasu in International Journal of Computers Communications & Control

<p> <img src="demo/2-src.png" width="200" alt="img"/> <img src="demo/2-gen.png" width="200" alt="img"/> <img src="demo/3-gen.png" width="200" alt="img"/> <img src="demo/3-gen.png" width="200" alt="img"/> </p>

Official PyTorch implementation of CainGAN, used to perform few-shot image generation. Starting from one or more source pictures, the model can synthesize face images in novel poses while preserving the person's identity.

License

Copyright (C) 2019 Andrei-Timotei Ardelean, Lucian Mircea Sasu

All rights reserved. Licensed under the CC BY-NC-SA 4.0 (Attribution-NonCommercial-ShareAlike 4.0 International)

Demo

Sources

<p> <img src="demo/1-src.png" width="200" alt="img"/> <img src="demo/6-src.png" width="200" alt="img"/> <img src="demo/4-src.png" width="200" alt="img"/> <img src="demo/5-src.png" width="200" alt="img"/> </p>

Generated (one shot)

<p> <img src="demo/1-gen.png" width="200" alt="img"/> <img src="demo/6-gen.png" width="200" alt="img"/> <img src="demo/4-gen.png" width="200" alt="img"/> <img src="demo/5-gen.png" width="200" alt="img"/> </p>

Image animation (one shot)

<p> <img src="demo/monalisa-src.jpg" width="200" alt="img"/> <img src="demo/monalisa.gif" width="200" alt="gif"/> <img src="demo/vincenzo-src.jpg" width="200" alt="img"/> <img src="demo/vincenzo.gif" width="200" alt="gif"/> </p>

Installation

A ready to use conda environment is provided. To create the python environment run:

conda env create -f environment.yaml
conda activate fewshot

To train the model you can use the train script.

python train.py --K 8 --cuda_visible 0 --data_root vox2selection/mp4 

This will train an 8-shot model on GPU 0. Change data_root path to the directory of your dataset. The format of the directory is expected to be similar to the one used in VoxCeleb2 dataset.
The extract_landmarks.py script can be used to precompute landmarks for faster training. For example:

python extract_landmarks.py --data_root vox2selection/mp4 --output_path vox2selection/land --device 'cuda'

Acknowledgement

Special thanks go to Xperi Corporation that provided the environment and physical resources that made this work possible.

Citation

Should you find this work useful for your research, please cite:

@article{IJCCC3862,
	author = {Andrei-Timotei Ardelean and Lucian Sasu},
	title = {Pose Manipulation with Identity Preservation},
	journal = {International Journal of Computers Communications & Control},
	volume = {15},
	number = {2},
	year = {2020},
	issn = {1841-9844},
	doi = {10.15837/ijccc.2020.2.3862},
	url = {http://univagora.ro/jour/index.php/ijccc/article/view/3862}
}