DMENet

DMENet: Deep Defocus Map Estimation Network<br>_{Official Implementation of the CVPR 2021 Paper}<br>_{Project | Paper | Supp | Poster}<br>

This repository contains the official matlab implementation of SYNDOF generation used in the following paper:

Deep Defocus Map Estimation using Domain Adaptation<br> Junyong Lee¹, Sungkil Lee², Sunghyun Cho³, and Seungyong Lee¹<br> ¹POSTECH, ²Sungkyunkwan University, ³DGIST<br> IEEE Computer Vision and Pattern Recognition (CVPR) 2019<br>

<p align="left"> <a href="https://junyonglee.me/projects/DMENet"> <img width=85% src="./assets/teaser_DMENet.gif" /> </a> </p>

Getting Started

Prerequisites

Tested environment

1. Setup environment

   • Option 1. install from scratch

     $ git clone https://github.com/codeslake/DMENet.git
     $ cd DMENet
     
     # for CUDA10
     $ conda create -y --name DMENet python=3.6 && conda activate DMENet
     $ sh install_CUDA10.0.sh
     
     # for CUDA11 (the name of conda environment matters)
     $ conda create -y --name DMENet_CUDA11 python=3.6 && conda activate DMENet_CUDA11
     $ sh install_CUDA11.1.sh
     

   • Option 2. docker

     $ nvidia-docker run --privileged --gpus=all -it --name DMENet --rm codeslake/dmenet:CVPR2019 /bin/zsh
     $ git clone https://github.com/codeslake/DMENet.git
     $ cd DMENet
     
     # for CUDA10
     $ conda activate DMENet
     
     # for CUDA11
     $ conda activate DMENet_CUDA11
     

2. Download and unzip datasets (OneDrive | Dropbox) under [DATASET_ROOT].

   [DATASET_ROOT]
    ├── train
    │   ├── SYNDOF
    │   ├── CUHK
    │   └── Flickr
    └── test
        ├── CUHK
        ├── RTF
        └── SYNDOF
   

   Note:

   • [DATASET_ROOT] is currently set to ./datasets/. It can be specified by modifying config.data_offset in ./config.py.

3. Download pretrained weights of DMENet (OneDrive | Dropbox) and unzip it as in [LOG_ROOT]/DMENet_CVPR2019/DMENet_BDCS/checkpoint/DMENet_BDCS.npz ([LOG_ROOT] is currently set to ./logs/).

4. Download pretrained VGG19 weights (OneDrive | Dropbox) and unzip as in pretrained/vgg19.npy (for training only).

Logs

• Training and testing logs will be saved under [LOG_ROOT]/DMENet_CVPR2019/[mode]/:

  [LOG_ROOT]
   └──DMENet_CVPR2019
      ├── [mode]
      │   ├── checkpoint      # model checkpoint
      │   ├── log             # scalar/image log for tensorboard
      │   ├── sample          # sample images of training
      │   └── result          # resulting images of evaluation
      └── ...
  

  [LOG_ROOT] can be modified with config.root_offset in ./config.py.

Testing final model of CVPR 2019

Please note that due to the server issue, the checkpoint used for the paper is lost. <br/>The provided checkpoint is the new checkpoint that shows the closest evaluation results as in the paper.

Check out updated performance with the new checkpoint.

• Test the final model by:

  python main.py --mode DMENet_BDCS --test_set CUHK
  

  Testing results will be saved in [LOG_ROOT]/DMENet_CVPR2019/[mode]/result/[test_set]/:

  ...
  [LOG_ROOT]/DMENet_CVPR2019/[mode]/result/
   └── [test_set]
       ├── image                     # input defocused images
       ├── defocus_map               # defocus images (network's direct output in range [0, 1])
       ├── defocus_map_min_max_norm  # min-max normalized defocus images in range [0, 1] for visualization
       └── sigma_map_7_norm          # sigma maps containing normalized standard deviations (in range [0, 1]) for a Gaussian kernel. For the actual standard deviation value, one should multiply 7 to this map.
  

  Quantitative results are computed from matlab. (e.g., evaluation on the RTF dataset).

  • Options
    • --mode: The name of a model to test. The logging folder named with the [mode] will be created as [LOG_ROOT]/DMENet_CVPR2019/[mode]/. Default: DMENet_BDCS
    • --test_set: The name of a dataset to evaluate. CUHK | RTF0 | RTF1 | RTF1_6 | random. Default: CUHK
      • The folder structure can be modified in the function get_eval_path(..) in ./config.py.
      • random is for testing models with any images, which should be placed as [DATASET_ROOT]/test/random/*.[jpg|png].

• Check out the evaluation code for the RTF dataset, and the deconvolution code.

Training & testing the network

• Train the network by:

  python main.py --is_train --mode [mode]
  

  Note:

  • If you train DMENet with newly generated SYNDOF dataset from this repo, comment this line and uncomment this line before the training.

• Test the network by:

  python main.py --mode [mode] --test_set [test_set]
  

  • arguments
    • --mode: The name of a model to train. The logging folder named with the [mode] will be created as [LOG_ROOT]/DMENet_CVPR2019/[mode]/. Default: DMENet_BDCS
    • --is_pretrain: Pretrain the network with the MSE loss (True | False). Default: False
    • --delete_log: Deletes [LOG_ROOT]/DMENet_CVPR2019/[mode]/* before training begins (True | False). Default: False

Contact

Open an issue for any inquiries. You may also have contact with junyonglee@postech.ac.kr

Related Links

• CVPR 2021: Iterative Filter Adaptive Network for Single Image Defocus Deblurring [paper][code]
• ICCV 2021: Single Image Defocus Deblurring Using Kernel-Sharing Parallel Atrous Convolutions [paper][code]
• SYNDOF dataset generation repo [link]

License

<br> This software is being made available under the terms in the LICENSE file. Any exemptions to these terms require a license from the Pohang University of Science and Technology.

Citation

If you find this code useful, please consider citing:

@InProceedings{Lee2019DMENet,
    author    = {Junyong Lee and Sungkil Lee and Sunghyun Cho and Seungyong Lee},
    title     = {Deep Defocus Map Estimation Using Domain Adaptation},
    booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    year      = {2019}
}