Joint Transformer and Mamba Fusion for Multispectral Object Detection

Intro

Official Code for Joint Transformer and Mamba Fusion for Multispectral Object Detection.

Installation

Install libraries including numpy, pytorch, timm, mamba-ssm, etc. according to requirement.txt

Training

1. change the pretrain-weight and data cfg.
2. change the img2path and filter label

3. python train.py
   

Test

1. change the weights and data cfg
2. change the img2path and filter label

3. python val.py
   

Visualize

1. change the weights and data cfg
2. change the img2path and filter label

3. python result_vis.py
   

Dataset

The datasets and annotations used in this repo:
-FLIR [BaiDu Drive] (code:jtm6)
-LLVIP [BaiDu Drive] (code:jtm6)
-M³FD [BaiDu Drive](code:jtm6)

Weight

-FLIR [BaiDu Drive] (code:jtm6)
-LLVIP [BaiDu Drive] (code:jtm6)
-M³FD [BaiDu Drive] (code:jtm6)

Pretrain_YOLOv5_Weight

-YOLOv5 [BaiDu Drive] (code:jtm6)

Cite

If you find our model/method/dataset useful, please cite our work:

@article{li2025joint,
  title={Joint Transformer and Mamba fusion for multispectral object detection},
  author={Li, Chao and Peng, Xiaoming},
  journal={Image and Vision Computing},
  pages={105468},
  year={2025},
  publisher={Elsevier}
}