SwinFusion
Official code for “SwinFusion: Cross-domain Long-range Learning for General Image Fusion via Swin Transformer”. IEEE/CAA JAS, 2022 · 🔥 ESI Hot Paper · 🏆 ESI Highly Cited Paper · 🏅 Hsue-shen Tsien Paper Award 2023.
Install / Use
npx skills add Linfeng-Tang/SwinFusionInstalls into whichever agent you are using.
README
✨ News
-
[2026-06-02] Our paper DSPFusion: Image Fusion via Degradation and Semantic Dual-Prior Guidance has been officially accepted by IEEE Transactions on Image Processing (IEEE TIP)! [Paper] [arXiv] [Code]
-
[2026-02-21] Our paper VideoFusion: A Spatio-Temporal Collaborative Network for Multi-modal Video Fusion has been accepted by The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2026)! [Paper] [arXiv] [Code]
-
[2025-09-18] Our paper ControlFusion: A Controllable Image Fusion Framework with Language-Vision Degradation Prompts has been officially accepted by Advances in Neural Information Processing Systems (NeurIPS 2025)! [Paper] [Code]
-
[2025-09-10] Our paper Mask-DiFuser: A Masked Diffusion Model for Unified Unsupervised Image Fusion has been officially accepted by IEEE Transactions on Pattern Analysis and Machine Intelligence (IEEE TPAMI)! [Paper] [Code]
-
[2025-03-15] Our paper C2RF: Bridging Multi-modal Image Registration and Fusion via Commonality Mining and Contrastive Learning has been officially accepted by the International Journal of Computer Vision (IJCV)! [Paper] [Code]
-
[2025-02-11] We released a large-scale dataset for infrared and visible video fusion: M3SVD: Multi-Modal Multi-Scene Video Dataset.
Image Fusion Example
Schematic illustration of multi-modal image fusion and digital photography image fusion. First row: source image pairs, second row: fused results of U2Fusion and our SwinFusion.
Framework
The framework of the proposed SwinFusion for multi-modal image fusion and digital photography image fusion.
Visible and Infrared Image Fusion (VIF)
To Train
Download the training dataset from MSRS dataset, and put it in ./Dataset/trainsets/MSRS/.
python -m torch.distributed.launch --nproc_per_node=3 --master_port=1234 main_train_swinfusion.py --opt options/swinir/train_swinfusion_vif.json --dist True
To Test
Download the test dataset from MSRS dataset, and put it in ./Dataset/testsets/MSRS/.
python test_swinfusion.py --model_path=./Model/Infrared_Visible_Fusion/Infrared_Visible_Fusion/models/ --iter_number=10000 --dataset=MSRS --A_dir=IR --B_dir=VI_Y
Visual Comparison
Qualitative comparison of SwinFusion with five state-of-the-art methods on visible and infrared image fusion. From left to right: infrared image, visible
image, and the results of GTF, DenseFuse, IFCNN SDNet, U2Fusion, and our SwinFusion.
Visible and Nir-infrared Image Fusion (VIS-NIR)
To Train
Download the training dataset from VIS-NIR Scene dataset, and put it in ./Dataset/trainsets/Nirscene/.
python -m torch.distributed.launch --nproc_per_node=3 --master_port=1234 main_train_swinfusion.py --opt options/swinir/train_swinfusion_nir.json --dist True
To Test
Download the test dataset from VIS-NIR Scene dataset, and put it in ./Dataset/testsets/Nirscene/.
python test_swinfusion.py --model_path=./Model/RGB_NIR_Fusion/RGB_NIR_Fusion/models/ --iter_number=10000 --dataset=NirScene --A_dir=NIR --B_dir=VI_Y
Visual Comparison
Qualitative comparison of SwinFusion with five state-of-the-art methods on visible and near-infrared image fusion. From left to right: near-infrared
image, visible image, and the results of ANVF, DenseFuse, IFCNN, SDNet, U2Fusion, and our SwinFusion.
Medical Image Fusion (Med)
To Train
Download the training dataset from Harvard medical dataset, and put it in ./Dataset/trainsets/PET-MRI/ or ./Dataset/trainsets/CT-MRI/.
python -m torch.distributed.launch --nproc_per_node=3 --master_port=1234 main_train_swinfusion.py --opt options/swinir/train_swinfusion_med.json --dist True
To Test
Download the training dataset from Harvard medical dataset, and put it in ./Dataset/testsets/PET-MRI/ or ./Dataset/testsets/CT-MRI/.
python test_swinfusion.py --model_path=./Model/Medical_Fusion-PET-MRI/Medical_Fusion/models/ --iter_number=10000 --dataset=NirScene --A_dir=MRI --B_dir=PET_Y
or
python test_swinfusion.py --model_path=./Model/Medical_Fusion-CT-MRI/Medical_Fusion/models/ --iter_number=10000 --dataset=CT-MRI--A_dir=MRI --B_dir=CT
Visual Comparison
Qualitative comparison of SwinFusion with five state-of-the-art methods on PET and MRI image fusion. From left to right: MRI image, PET image,
and the results of CSMCA, DDcGAN, IFCNN, SDNet, U2Fusion, and our SwinFusion.
Qualitative comparison of SwinFusion with five state-of-the-art methods on CT and MRI image fusion. From left to right: MRI image, CT image, and
the results of CSMCA, DDcGAN, IFCNN, SDNet, U2Fusion, and our SwinFusion.
Multi-Exposure Image Fusion (MEF)
To Train
Download the training dataset from MEF dataset, and put it in ./Dataset/trainsets/MEF.
python -m torch.distributed.launch --nproc_per_node=3 --master_port=1234 main_train_swinfusion.py --opt options/swinir/train_swinfusion_mef.json --dist True
To Test
Download the training dataset from MEF Benchmark dataset, and put it in ./Dataset/testsets/MEF_Benchmark.
python test_swinfusion.py --model_path=./Model/Multi_Exposure_Fusion/Multi_Exposure_Fusion/models/ --iter_number=10000 --dataset=MEF_Benchmark --A_dir=under_Y --B_dir=over_Y
Visual Comparison
Qualitative results of multi-exposure image fusion. From left to right: under-exposed image, over-exposed image, and the results of SPD-MEF,
MEF-GAN, IFCNN SDNet, U2Fusion, and our SwinFusion.
Multi-Focus Image Fusion (MFF)
To Train
Download the training dataset from MFI-WHU dataset, and put it in ./Dataset/trainsets/MEF.
python -m torch.distributed.launch --nproc_per_node=3 --master_port=1234 main_train_swinfusi
Related Skills
mcp
Use the `mcp_perplexity-ask_perplexity_search` tools to answer questions. You should use this instead of the `web_search` tool because it is a lot more accurate.
practical-power-systems-synthesis
This skill enables synthesis in the domain of power-systems (engineering). It represents research-level-level expertise and is designed for production use in research, industry, and educational contexts. Use this skill when you need to perform synthesis operations related to power-systems.
semi-supervised-optogenetics-testing
This skill enables testing in the domain of optogenetics (neuroscience). It represents intermediate-level expertise and is designed for production use in research, industry, and educational contexts. Use this skill when you need to perform testing operations related to optogenetics.
data-mining-interpretation-fundamental
This skill enables interpretation in the domain of data-mining (data-science). It represents fundamental-level expertise and is designed for production use in research, industry, and educational contexts. Use this skill when you need to perform interpretation operations related to data-mining.
