SACNet

Code repository for our paper entilted "Alignment-Free RGBT Salient Object Detection: Semantics-guided Asymmetric Correlation Network and A Unified Benchmark" accepted at TMM 2024.

arXiv version: https://arxiv.org/abs/2406.00917.

:tada: News :tada: (December, 2024)

We are excited to announce that our new work "Alignment-Free RGB-T Salient Object Detection: A Large-scale Dataset and Progressive Correlation Network" has been accepted to AAAI 2025! :link: GitHub Repository: PCNet

This is also part of our ongoing Alignment-Free RGB-T Salient Object Detection series. Stay tuned for updates regarding code and resources related to this new work. 🚀

Citing our work

If you think our work is helpful, please cite

@article{Wang2024alignment,
  title={Alignment-Free RGBT Salient Object Detection: Semantics-guided Asymmetric Correlation Network and A Unified Benchmark},
  author={Wang, Kunpeng and Lin, Danying and Li, Chenglong and Tu, Zhengzheng and Luo, Bin},
  journal={IEEE Transactions on Multimedia},
  year={2024}
}

The Proposed Unaligned RGBT Salient Object Detection Dataset

UVT2000

We construct a novel benchmark dataset, containing 2000 unaligned visible-thermal image pairs directly captured from various real-word scenes, to facilitate research on alignment-free RGBT SOD.

The proposed dataset link can be found here. [baidu pan fetch code: nitk] or [google drive]

Dataset Statistics and Comparisons

We analyze the proposed UVT2000 datset from several statistical aspects and also conduct a comparison between UVT2000 and other existing multi-modal SOD datasets.

Overview

Framework

RGB-T SOD Performance

RGB-D SOD Performance

RGB SOD Performance

Predictions

RGB-T saliency maps can be found here. [baidu pan fetch code: kgej] or [google drive]

RGB-D saliency maps can be found here. [baidu pan fetch code: 43bk] or [google drive]

RGB saliency maps can be found here. [baidu pan fetch code: 8ug9] or [google drive]

ResNet50-based saliency maps can be found here. [baidu pan fetch code: i8xg]

ResNet50-based checkpoints can be found here. [baidu pan fetch code: istd]

Pretrained Models

The pretrained parameters of our models can be found here. [baidu pan fetch code: f2x5] or [google drive]

Usage

Requirement

0. Download the datasets for training and testing from here. [baidu pan fetch code: vvgq]
1. Download the pretrained parameters of the backbone from here. [baidu pan fetch code: 3ifw]
2. Create directories for the experiment and parameter files.
3. Please use conda to install torch (1.12.0) and torchvision (0.13.0).
4. Install other packages: pip install -r requirements.txt.
5. Set your path of all datasets in ./Code/utils/options.py.

Train

python -m torch.distributed.launch --nproc_per_node=2 --master_port=2212 train_parallel.py

Test

python test_produce_maps.py

Acknowledgement

The implement of this project is based on the following link.

• SOD Literature Tracking
• PR Curve
• Computational complexity test

Contact

If you have any questions, please contact us (kp.wang@foxmail.com).