Reverse Convolution and Its Application to Image Restoration

Xuhong Huang^*, Shiqi Liu^*, Kai Zhang^†, Ying Tai, Jian Yang, Hui Zeng, Lei Zhang

Nanjing University, China

---

• Motivation
• Method
• Models
• Codes
• PSNR results
• Visual results of Converse-DnCNN
• Visual results of Converse-SRResNet
• Visual results of ConverseNet
• Visual results of Converse-USRNet

Motivation

Convolution and transposed convolution (often referred to as deconvolution) are fundamental operations in deep neural networks. Convolution is commonly used for feature extraction and spatial downsampling. In contrast, transposed convolution is used to upsample spatial dimensions. Due to this functional relationship, transposed convolution is sometimes described in the literature as a reverse convolution operator. However, it is not the mathematical inverse of convolution. Instead, it performs upsampling by inserting zeros between input elements, followed by a standard convolution. While this interpretation is widely accepted, implementing a reverse convolution operator has received little attention. Notably, popular deep learning frameworks such as PyTorch do not provide native support for such an operator.

<img src="figs/image.png" width="500px"/>

Method

• We propose a novel depthwise reverse convolution operator as a first-step exploration to effectively reverse the depthwise convolution by formulating and solving a regularized least-squares optimization problem. We thoroughly investigate its kernel initialization, padding strategies, and other critical aspects to ensure its effective implementation.

<img src="figs/algorithm.png" width="400px"/> <!-- $$ \mathbf{Y} = (\mathbf{X} \otimes \mathbf{K})\downarrow_{s}. $$ $$ \mathbf{X} = \mathcal{F}(\mathbf{Y}, \mathbf{K}, s). $$ $$ \mathbf{X}^\ast = \arg\min_{\mathbf{X}} \left\| \mathbf{Y} - \left( \mathbf{X} \otimes \mathbf{K} \right) \downarrow_{s} \right\|_F^2 + \lambda \left\| \mathbf{X} - \mathbf{X}_0 \right\|_F^2, $$ $$ \mathbf{X}^\ast = \arg\min_{\mathbf{X}} \left\| \mathbf{Y} - \left( \mathbf{X} \otimes \mathbf{K} \right) \downarrow_{s} \right\|_F^2 $$ $$ \mathbf{X}^{*} = \mathbf{F}^{-1}\!\left(\frac{1}{\lambda}\Big(\mathbf{L} - \overline{\mathbf{F}_K} \odot_{s}\frac{(\mathbf{F}_K\mathbf{L})\Downarrow_{{s}} }{|\mathbf{F}_K|^2\Downarrow_{{s}} +\lambda}\Big)\right) $$ -->

• Building upon this reverse convolution operator, we integrate it with layer normalization, 1x1 convolution, and GELU activation to form a reverse convolution block, similar to a Transformer block.

<img src="figs/block.png" width="400px"/>

Models

|Model|# params| |---|:---:| |Converse-DnCNN | 734,913 | |Converse-SRResNet | 694,208 | |ConverseNet | 243,267 | |Converse-USRNet | 307,091 |

Codes

• main_test_converse_dncnn.py: Code to denoise images by Converse-DnCNN
• main_test_converse_srresnet.py: Code to super-resolve LR images by Converse-SRResNet
• main_test_converse_usrnet.py: Code to super-resolve LR images by Converse-USRNet

PSNR results

Run main_test_converse_dncnn.py to produce the following results.

<img src="figs/psnr_dncnn.png" width="500px"/> The table shows the number of parameters and average PSNR(dB) results of different models for Gaussian denoising with noise level 25 on Set12 and BSD68 datasets.

Run main_test_converse_srresnet.py to produce the following results.

<img src="figs/psnr_srresnet.png" width="500px"/> The table shows the average PSNR(dB) results of different variants of SRResNet for super-resolution with scale factor 4.

Run main_test_converse_usrnet.py to produce the following results.

<img src="figs/psnr_usrnet.png" width="500px"/> The table shows the average PSNR(dB) results of different methods on Set5, Set14, BSD100 and Urban100 datasets.

Visual results of Converse-DnCNN

|<img src="figs/noisy.png" width="430x"/> |<img src="figs/reverse_23.53.png" width="430x"/>| |:---:|:---:| |<i>Image with noise level uniformly ranging from 0 to 50</i>|<i>Denoising result by Converse-DnCNN</i>|

Visual results of Converse-SRResNet

|<img src="figs/downsample.png" width="430x"/> |<img src="figs/SR_reverse.png" width="430x"/>| |:---:|:---:| |<i>LR image</i>|<i>Super-resolution result by Converse-SRResNet</i>|

Visual results of ConverseNet

|<img src="figs/img_088_LR_12_output.png" width="430x"/> |<img src="figs/img_088_denoiseconverse_12_output.png" width="430x"/>| |:---:|:---:| |<i>Blurred image</i>|<i>Deblurring results by ConverseNet</i>|

Visual results of Converse-USRNet

|<img src="figs/img_022_LR_12_output.png" width="430x"/> |<img src="figs/img_022_standard_12_output.png" width="430x"/>| |:---:|:---:| |<i>Blurred image</i>|<i>Deblurring results by Converse-USRNet</i>|

Citation

@inproceedings{huang2025reverse,
    title={Reverse Convolution and Its Applications to Image Restoration},
    author={Huang, Xuhong and Liu, Shiqi and Zhang, Kai and Tai, Ying and Yang, Jian and Zeng, Hui and Zhang, Lei},
    booktitle={International Conference on Computer Vision},
    pages={1--10},
    year={2025}
}