ResULIC

<!-- ## Ultra Lowrate Image Compression with Semantic Residual Coding and Compression-aware Diffusion --> <h1 align="center"> Ultra Lowrate Image Compression with Semantic Residual Coding and Compression-aware Diffusion </h1> <p align="center"> Anle&nbsp;Ke¹ &ensp; <b>&middot;</b> &ensp; Xu&nbsp;Zhang¹ &ensp; <b>&middot;</b> &ensp; Tong&nbsp;Chen¹ &ensp; <b>&middot;</b> &ensp; Ming&nbsp;Lu¹&ensp; <b>&middot;</b> &ensp; Chao&nbsp;Zhou²&ensp; <b>&middot;</b> &ensp; Jiawen&nbsp;Gu²&ensp; <b>&middot;</b> &ensp; Zhan&nbsp;Ma¹&ensp; </b> &ensp; </p> <p align="center"> ¹ Nanjing University &emsp; ²Kuaishou Technology&emsp; </p> <p align="center"> <a href="https://njuvision.github.io/ResULIC/">🌐 Project Page</a> &ensp; <a href="https://arxiv.org/abs/2505.08281">📃 Paper</a> &ensp; </p> <p align="center"> <img src="./fig/arc.png" style="border-radius: 15px"><br> </p>

:book: Table Of Contents

• ✨ Visual Results
• ⏳ Train
• 😀 Inference
• 🌊 TODO
• ❤ Acknowledgement
• 🙇‍ Citation

⚙️ Environment Setup

- conda create -n ResULIC python=3.10
- conda activate ResULIC
- pip install -r requirements.txt

<a name="Visual Results"></a>✨ Visual Results

<p align="center"> <img src="./fig/visual.png" style="border-radius: 15px"><br> </p>

<a name="train"></a>⏳ Train

Note: The numbers in the yaml filenames (e.g., 1_1_1) represent $\lambda_{\text{diffusion}}$, $\lambda_{\text{mse}}$, and $\lambda_{\text{bpp}}$ respectively.

Stage 1: Initial Training

1. Download Pretrained Model
   Download the pretrained Stable Diffusion v2.1 model into the ./weight directory:

   wget https://huggingface.co/stabilityai/stable-diffusion-2-1-base/resolve/main/v2-1_512-ema-pruned.ckpt --no-check-certificate -P ./weight
   
   

2. Modify the configuration file./configs/train_zc_eps.yaml and ./configs/model/stage1/xx.yaml accordingly.

3. Start training.

   bash stage1.sh 
   

Stage 2:

1. Modify the configuration file ./configs/train_stage2.yaml and ./configs/model/stage2/xx.yaml accordingly.

2. Start training.

   bash stage2.py 
   

<a name="inference"></a>😀 Inference

1. Download the pre-trained models (We provide 3 low-bitrate models for testing, and other bitrate models can be trained by referring to the training method):

   | Bitrate | Link | | --------- | ---------- | | 0.03 bpp | stage2/1_1_4/300 | | 0.01 bpp | stage2/1_1_12/400 | | 0.002 bpp | stage2/1_1_24/600 |

Note: It is recommended to set "ddim_steps" to a number that is divisible by "add_steps". For example, when add_steps=600, ddim_steps could be 2, 3, 5...

1. W/o Srr, W/o Pfo.

   CUDA_VISIBLE_DEVICES=2 python inference_win.py \
    --ckpt xx \
    --config /xx/xx.yaml \
    --output xx/ \
    --ddim_steps 3 \
    --ddim_eta 0 \
    --Q x.0 \
    --add_steps x00
   

2. W/ Srr, W/o Pfo.

    CUDA_VISIBLE_DEVICES=2 python inference_res.py \
    --ckpt xx \
    --config /xx/xx.yaml \
    --output xx/ \
    --ddim_steps 3 \
    --ddim_eta 0 \
    --Q x.0 \
    --add_steps x00
   

3. W/ Srr, W/ Pfo.

    CUDA_VISIBLE_DEVICES=2 python inference_res_pfo.py \
    --ckpt xx \
    --config /xx/xx.yaml \
    --output xx/ \
    --ddim_steps x \
    --ddim_eta 0 \
    --Q x.0 \
    --add_steps x00
   

<a name="todo"></a>🌊 TODO

• [x] Release code
• [x] Release quantitative metrics （👾The quantitative metrics for ResULIC presented in our paper can be found in indicator.）
• [x] Release pretrained models

<a name="acknowledgement">❤ Acknowledgement

This work is based on ControlNet, ControlNet-XS, DiffEIC, and ELIC, thanks to their invaluable contributions.

<a name="cite"></a>🙇‍ Citation

If you find our work useful, please consider citing:

@inproceedings{Ke2025resulic,
               author = {Ke, Anle and Zhang, Xu and Chen, Tong and Lu, Ming and Zhou, Chao and Gu, Jiawen and Ma, Zhan},
               title = {Ultra Lowrate Image Compression with Semantic Residual Coding and Compression-aware Diffusion},
               booktitle = {International Conference on Machine Learning},
               year = {2025}
               }