S2Dnet

Specular-to-Diffuse Translation for Multi-View Reconstruction <br/> Shihao Wu¹, Hui Huang², Tiziano Portenier¹, Matan Sela³, Daniel Cohen-Or⁴, Ron Kimmel³, and Matthias Zwicker⁵    <br/> ¹University of Bern, ²Shenzhen University, ³Technion - Israel Institute of Technology, ⁴Tel Aviv University, ⁵University of Maryland <br/> European Conference on Computer Vision (ECCV), 2018

<p align="center"><img width="80%" src="git_img/teaser.png" /></p> <p align="center"><img width="100%" src="git_img/network.png" /></p> <br/> <br/>

Dependencies

• Python
• PyTorch
• TensorFlow (optional for tensorboard)
• PBRT (optional for rendering)
• CycleGAN (our code is based on this implementation)

Update 10/April/2019: The code has been updated to pytorch 0.4. A single-view synthetic dataset (75 GB) is provided, one can train pix2pix or cycleGAN on it.

To-do list:

• Implement a single-view translation network (with multi-scale discriminator, re-convolution and pixel-normalization) and provide a testing script.

Downloading (Dropbox links)

• Large single-view synthetic training data (75 GB)

• Large multi-view synthetic training data (172 GB)

• Small multi-view synthetic training data (3 GB)

• Some single-view synthetic training data (40 GB)

• Real training data (7GB)

• Some Synthetic testing data (1 GB)

• Some Real testing data

• Aligned 3D models for rendering (1.8 GB)

• Environment maps for rendering

• Pretrained networks using pix2pix

• Pretrained networks using cycleGAN

• Pretrained networks using S2Dnet

• Some Reconstruction results of Real Data

• Some Reconstruction results of Synthetic Data

Training example

$ python train_multi_view.py --dataroot ../huge_uni_render_rnn --logroot ./logs/job101CP --name job_submit_101C_re1_pixel --model cycle_gan --no_dropout --loadSize 512 --fineSize 512 --patchSize 256 --which_model_netG unet_512_Re1 --which_model_netD patch_512_256_multi_new --lambda_A 10 --lambda_B 10 --lambda_vgg 5 --norm pixel

Testing

Please refer to "./useful_scripts/evaluation/"

Scripts of SIFT, SMVS, and rendering are in "./useful_scripts/".

Please contact the author for more information about the code and data.