Video Super Resolution

A collection of state-of-the-art video or single-image super-resolution architectures, reimplemented in tensorflow.

Project uploaded to PyPI now. Try install from PyPI:

pip install VSR

Pretrained weights is uploading now.

Several referenced PyTorch implementations are also included now.

Quick Link:

• Installation
• Getting Started
• Benchmark

Network list and reference (Updating)

The hyperlink directs to paper site, follows the official codes if the authors open sources.

All these models are implemented in ONE framework.

|Model |Published |Code* |VSR (TF)**|VSR (Torch)|Keywords|Pretrained| |:-----|:---------|:-----|:---------|:----------|:-------|:---------| |SRCNN|ECCV14|-, Keras|Y|Y| Kaiming |√| |RAISR|arXiv|-|-|-| Google, Pixel 3 || |ESPCN|CVPR16|-, Keras|Y|Y| Real time |√| |VDSR|CVPR16|-|Y|Y| Deep, Residual |√| |DRCN|CVPR16|-|Y|Y| Recurrent || |DRRN|CVPR17|Caffe, PyTorch|Y|Y| Recurrent || |LapSRN|CVPR17|Matlab|Y|-| Huber loss || |EDSR|CVPR17|-|Y|Y| NTIRE17 Champion |√| |SRGAN|CVPR17|-|Y|-| 1st proposed GAN || |VESPCN|CVPR17|-|Y|Y| VideoSR |√| |MemNet|ICCV17|Caffe|Y|-||| |SRDenseNet|ICCV17|-, PyTorch|Y|-| Dense |√| |SPMC|ICCV17|Tensorflow|T|Y| VideoSR || |DnCNN|TIP17|Matlab|Y|Y| Denoise |√| |DCSCN|arXiv|Tensorflow|Y|-||| |IDN|CVPR18|Caffe|Y|-| Fast |√| |RDN|CVPR18|Torch|Y|-| Deep, BI-BD-DN || |SRMD|CVPR18|Matlab|-|Y| Denoise/Deblur/SR |√| |DBPN|CVPR18|PyTorch|Y|Y| NTIRE18 Champion |√| |ZSSR|CVPR18|Tensorflow|-|-| Zero-shot || |FRVSR|CVPR18|PDF|T|Y| VideoSR |√| |DUF|CVPR18|Tensorflow|T|-| VideoSR || |CARN|ECCV18|PyTorch|Y|Y| Fast |√| |RCAN|ECCV18|PyTorch|Y|Y| Deep, BI-BD-DN || |MSRN|ECCV18|PyTorch|Y|Y| |√| |SRFeat|ECCV18|Tensorflow|Y|Y| GAN || |NLRN|NIPS18|Tensorflow|T|-| Non-local, Recurrent || |SRCliqueNet|NIPS18|-|-|-| Wavelet || |FFDNet|TIP18|Matlab|Y|Y| Conditional denoise|| |CBDNet|CVPR19|Matlab|T|-| Blind-denoise || |SOFVSR|ACCV18|PyTorch|-|Y| VideoSR |√| |ESRGAN|ECCVW18|PyTorch|-|Y|1st place PIRM 2018|√| |TecoGAN|arXiv|Tensorflow|-|T| VideoSR GAN|√| |RBPN|CVPR19|PyTorch|-|Y| VideoSR |√| |DPSR|CVPR19|Pytorch|-|-||| |SRFBN|CVPR19|Pytorch|-|-|||| |SRNTT|CVPR19|Tensorflow|-|-|Adobe|| |SAN|CVPR19|empty|-|-| AliDAMO SOTA || |AdaFM|CVPR19|Pytorch|-|-| SenseTime Oral ||

*The 1st repo is by paper author.

**Y: included; -: not included; T: under-testing.

You can download pre-trained weights through prepare_data, or visit the hyperlink at √.

Link of datasets

(please contact me if any of links offend you or any one disabled)

|Name|Usage|#|Site|Comments| |:---|:----|:----|:---|:-----| |SET5|Test|5|download|jbhuang0604| |SET14|Test|14|download|jbhuang0604| |SunHay80|Test|80|download|jbhuang0604| |Urban100|Test|100|download|jbhuang0604| |VID4|Test|4|download|4 videos| |BSD100|Train|300|download|jbhuang0604| |BSD300|Train/Val|300|download|-| |BSD500|Train/Val|500|download|-| |91-Image|Train|91|download|Yang| |DIV2K|Train/Val|900|website|NTIRE17| |Waterloo|Train|4741|website|-| |MCL-V|Train|12|website|12 videos| |GOPRO|Train/Val|33|website|33 videos, deblur| |CelebA|Train|202599|website|Human faces| |Sintel|Train/Val|35|website|Optical flow| |FlyingChairs|Train|22872|website|Optical flow| |DND|Test|50|website|Real noisy photos| |RENOIR|Train|120|website|Real noisy photos| |NC|Test|60|website|Noisy photos| |SIDD(M)|Train/Val|200|website|NTIRE 2019 Real Denoise| |RSR|Train/Val|80|download|NTIRE 2019 Real SR| |Vimeo-90k|Train/Test|89800|website|90k HQ videos|

Other open datasets: Kaggle ImageNet COCO

VSR package

This package offers a training and data processing framework based on TF. What I made is a simple, easy-to-use framework without lots of encapulations and abstractions. Moreover, VSR can handle raw NV12/YUV as well as a sequence of images as inputs.

Install

1. Prepare proper tensorflow and pytorch(optional). For example, GPU and CUDA10.0 (recommend to use conda)