📖 Spatially-Adaptive Feature Modulation for Efficient Image Super-Resolution

<a href="https://colab.research.google.com/drive/19DdsNFeOYR8om8QCCi9WWzr_WkWTLHZd?usp=sharing"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="google colab logo"></a> [Paper] [Supp]

Long Sun, Jiangxin Dong, Jinhui Tang, and Jinshan Pan IMAG Lab, Nanjing University of Science and Technology

An overview of the proposed SAFMN. SAFMN first transforms the input LR image into the feature space using a convolutional layer, performs feature extraction using a series of feature mixing modules (FMMs), and then reconstructs these extracted features by an upsampler module. The FMM block is mainly implemented by a spatially-adaptive feature modulation (SAFM) layer and a convolutional channel mixer (CCM).

🚩 News

[2025-12-23] Add the Real_SAFMN++, which is the 1st place winner of the AIM 2025 Efficient Perceptual Super-Resolution Challenge.
[2024-07-16] Add 🤗HuggingFace online demo!
[2024-05-08] Add the light_SAFMN++, which is the 1st place winner of the fidelity track of the Real-time 4K Super Resolution Challenge for compressed AVIF images, and is invited to give an oral presentation at the AIS2024 workshop.
[2024-03-27] Add the improved ESR model SAFMN++, which ranks Top4 in the Overall Performance track and Runtime track of NTIRE2024 ESR challenge
[2024-03-16] Add SAFMN_BCIE, which enhances the quality of JPEG images compressed with a large range of quality factors. The corresponding inference code can be found here.
[2023-11-22] The code for ONNX export is available here
[2023-09-08] Add :panda_face: OpenXLab online demo!
[2023-08-31] Update SAFMN_Real_x4.pth
[2023-08-31] Add SAFMN_Real_x2.pth
[2023-08-21] Colab demo for SAFMN on x4 real-world image SR is available <a href="https://colab.research.google.com/drive/19DdsNFeOYR8om8QCCi9WWzr_WkWTLHZd?usp=sharing"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="google colab logo"></a>
[2023-07-14] Our SAFMN is accepted to ICCV 2023
[2023-06-06] The report of NTIRE 2023 Challenge on Efficient Super-Resolution is available here
[2023-04-17] The SAFMN ranks Top6 for overall performance in the NTIRE2023 ESR challenge.
[2023-04-17] The SAFMN variant ranks Top3 for model complexity in the NTIRE2023 ESR Challenge.
[2023-03-22] The code and checkpoint for the NTIRE2023 Efficient Super-Resolution Challenge is available here.
[2023-03-22] The pre-trained model with high-order degradation on the LSDIR dataset is available.
[2023-03-13] The source codes, checkpoints and visual results are available.
[2023-02-26] The paper is available here.

👀 Demos

Real-world SR Results <img src="figs/real_figs/anime_results.png">

|Real-World Image (x4)|Real-ESRGAN |SwinIR | SAFMN (ours)| | :--- | :---: | :-----: | :-----: |
| <img width="350" src="figs/real_figs/five_golden_flowers_02.png">|<img width="350" src="figs/real_figs/five_golden_flowers_02_realESRGAN.png">|<img width="350" src="figs/real_figs/five_golden_flowers_02_SwinIR.png">|<img width="350" src="figs/real_figs/five_golden_flowers_02_SAFMN.png"> | <img width="350" src="figs/real_figs/five_golden_flowers_01.png">|<img width="350" src="figs/real_figs/five_golden_flowers_01_realESRGAN.png">|<img width="350" src="figs/real_figs/five_golden_flowers_01_SwinIR.png">|<img width="350" src="figs/real_figs/five_golden_flowers_01_SAFMN.png"> | <img width="350" src="figs/real_figs/kobe_curry.png">|<img width="350" src="figs/real_figs/kobe_curry_realESRGAN.png">|<img width="350" src="figs/real_figs/kobe_curry_SwinIR.png">|<img width="350" src="figs/real_figs/kobe_curry_SAFMN.png"> | <img width="350" src="figs/real_figs/little_carp.png">|<img width="350" src="figs/real_figs/little_carp_realESRGAN.png">|<img width="350" src="figs/real_figs/little_carp_SwinIR.png">|<img width="350" src="figs/real_figs/little_carp_SAFMN.png">

Results on Blind Compressed Images

Results on AIGC Image

🔧 Requirements and Installation

Python 3.8, PyTorch >= 1.11

BasicSR 1.4.2

Platforms: Ubuntu 18.04, cuda-11

Installation

# Clone the repo
git clone https://github.com/sunny2109/SAFMN.git
# Install dependent packages
cd SAFMN
pip install -r requirements.txt
# Install BasicSR
python setup.py develop

You can also refer to this INSTALL.md for installation

Training and Testing

Training

Run the following commands for training:

# train SAFMN for x4 effieicnt SR
python basicsr/train.py -opt options/train/SAFMN/train_DF2K_x4.yml
# train SAFMN for x4 classic SR
python basicsr/train.py -opt options/train/SAFMN/train_L_DF2K_x4.yml

⚡ Quick Inference

Download the pretrained models.
Download the testing dataset.
Run the following commands:

# test SAFMN for x4 efficient SR
python basicsr/test.py -opt options/test/SAFMN/test_benchmark_x4.yml
# test SAFMN for x4 classic SR
python basicsr/test.py -opt options/test/SAFMN/test_L_benchmark_x4.yml
# test SAFMN for x4 real-world SR (without ground-truth)
python basicsr/test.py -opt options/test/SAFMN/test_real_img_x4.yml
# test SAFMN for x4 real-world SR (large input)
python inference/inference_real_safmn.py --input test_demo --output results/test_demo --scale 4 --large_input

The test results will be in './results'.

Pretrained Models and Results

Pretrained Models

We have provided three ways to download our checkpoints.
- 1.Download via the Google Drive links shown below.
- 2.Download via the Baidu Netdisk links shown below.
- 3.Visit our huggingface repo at https://huggingface.co/Meloo/SAFMN/tree/main and click the download icons.

SAFMN

Install / Use

README