Deep Cyclic Generative Adversarial Residual Convolutional Networks for Real Image Super-Resolution (SRResCycGAN)

An official PyTorch implementation of the SRResCycGAN network as described in the paper Deep Cyclic Generative Adversarial Residual Convolutional Networks for Real Image Super-Resolution. This work is participated in the AIM 2020 Real-Image Super-resolution challenge track-3 at the high x4 upscaling factor.

Abstract

Recent deep learning based single image super-resolution (SISR) methods mostly train their models in a clean data domain where the low-resolution (LR) and the high-resolution (HR) images come from noise-free settings (same domain) due to the bicubic down-sampling assumption. However, such degradation process is not available in real-world settings. We consider a deep cyclic network structure to maintain the domain consistency between the LR and HR data distributions, which is inspired by the recent success of CycleGAN in the image-to-image translation applications. We propose the Super-Resolution Residual Cyclic Generative Adversarial Network (SRResCycGAN) by training with a generative adversarial network (GAN) framework for the LR to HR domain translation in an end-to-end manner. We demonstrate our proposed approach in the quantitative and qualitative experiments that generalize well to the real image super-resolution and it is easy to deploy for the mobile/embedded devices. In addition, our SR results on the AIM 2020 Real Image SR Challenge datasets demonstrate that the proposed SR approach achieves comparable results as the other state-of-art methods.

Spotlight Video

Pre-trained Models

| Datasets|SRResCycGAN| |---|:---:| |NTIRE2020 RWSR|Sensor noise (σ = 8)| |NTIRE2020 RWSR|JPEG compression (quality=30)| |NTIRE2020 RWSR|Unknown corruptions| |AIM2020 RISR|Real image corruptions|

BibTeX

@InProceedings{Umer_2020_ECCVW,
               author = {Muhammad Umer, Rao and Micheloni, Christian},
               title = {Deep Cyclic Generative Adversarial Residual Convolutional Networks for Real Image Super-Resolution},
               booktitle = {Proceedings of the European Conference on Computer Vision (ECCV) Workshops},
               month = {August},
               year = {2020}
              }

Quick Test

This model can be run on arbitrary images with a Docker image hosted on Replicate: https://beta.replicate.ai/RaoUmer/SRResCycGAN. Below are instructions for how to run the model without Docker:

Dependencies

• Python 3.7 (version >= 3.0)
• PyTorch >= 1.0 (CUDA version >= 8.0 if installing with CUDA.)
• Python packages: pip install numpy opencv-python

Train models

• The SR training code is based on the SRResCGAN.

Test models

1. Clone this github repository as the following commands:

git clone https://github.com/RaoUmer/SRResCycGAN
cd SRResCycGAN
cd srrescycgan_code_demo

2. Place your own low-resolution images in the ./srrescycgan_code_demo/LR folder. (There are two sample images i.e. LR_006 and LR_014).
3. Download the pretrained models from Pre-trained Models section. Place the models in ./srrescycgan_code_demo/trained_nets_x4.
4. Run the test. You can config in the test_srrescycgan.py.

python test_srrescgan.py

5. The results are in the ./srrescycgan_code_demo/sr_results_x4 folder.

SRResCycGAN Architecture

Overall Representative diagram

<p align="center"> <img height="200" src="figs/srrescycgan.png"> </p>

Quantitative Results

The x4 SR quantitative results comparison of our method with others over the DIV2K validation-set (100 images). The best performance is shown in red and the second best performance is shown in blue.

<p align="center"> <img height="250" src="figs/res_quant_comparison.png"> </p>

The AIM2020 Real Image SR Challenge Results (x4)

| _Team | _PSNR↑ | _SSIM↑ | _{Weighed_score↑} | |:---:|:---:|:---:|:---:| | _Baidu| _31.3960 |_0.8751|_{0.7099 (1)}| | _ALONG| _31.2369 |_0.8742|_{0.7076 (2)}| | _CETC-CSKT| _31.1226 |_0.8744|_{0.7066 (3)}| | _SR-IM| _31.2369 |_0.8728|_0.7057| | _DeepBlueAI| _30.9638 |_0.8737|_0.7044| | _JNSR| _30.9988 |_0.8722|_0.7035| | _{OPPO_CAMERA}| _30.8603 |_0.8736|_0.7033| | _Kailos| _30.8659 |_0.8734|_0.7031| | _{SR_DL}| _30.6045 |_0.8660|_0.6944| | _{Noah_TerminalVision}| _30.5870 |_0.8662|_0.6944| | _Webbzhou| _30.4174 |_0.8673|_0.6936| | _{TeamInception}| _30.3465 |_0.8681|_0.6935| | _IyI| _30.3191 |_0.8655|_0.6911| | _MCML-Yonsei| _30.4201 |_0.8637|_0.6906| | _MoonCloud| _30.2827 |_0.8644|_0.6898| | _qwq| _29.5878 |_0.8547|_0.6748| | _SrDance| _29.5952 |_0.8523|_0.6729| | _{MLP_SR (ours)}| _28.6185 |_0.8314|_0.6457| | _EDSR| _28.2120 |_0.8240|_0.6356| | _{RRDN_IITKGP}| _27.9708 |_0.8085|_0.6201| | _congxiaofeng| _26.3915 |_0.8258|_0.6187|

Visual Results

DIV2K Validation-set (100 images)

Here are the SR resutls comparison of our method on the DIV2K validation-set images.

<p align="center"> <img height="300" src="figs/div2k_res_val.png"> </p>

Real-Image SR Challenge dataset images (Track-3)

Validation-set

You can download all the SR resutls of our method on the AIM 2020 Real-Image SR validation-set from the Google Drive: SRResCycGAN.

<p align="center"> <img height="300" src="figs/res_rwsr_val1.png"> </p> <p align="center"> <img height="300" src="figs/res_rwsr_val2.png"> </p>

Test-set

You can download all the SR resutls of our method on the AIM 2020 Real-Image SR test-set from the Google Drive: SRResCycGAN.

<p align="center"> <img height="300" src="figs/res_rwsr_test1.png"> </p> <p align="center"> <img height="300" src="figs/res_rwsr_test2.png"> </p>