<div align="center"> <img src="https://github.com/lartpang/PyIRSTDMetrics/blob/main/images/logo.png?raw=true" alt="Logo" width="320" height="auto"> </br> <h2>PyIRSTDMetrics: A simple and efficient implementation for the IRSTD performance analysis</h2> <a href='https://arxiv.org/abs/2509.16888'> <img src='https://img.shields.io/badge/Paper-arXiv-red?style=flat&logo=arXiv&logoColor=red' alt='arXiv'> </a> <img src="https://img.shields.io/pypi/v/pyirstdmetrics"> <img src="https://img.shields.io/pypi/dm/pyirstdmetrics?label=pypi%20downloads&logo=PyPI&logoColor=white"> <img src="https://img.shields.io/github/last-commit/lartpang/PyIRSTDMetrics"> <img src="https://img.shields.io/github/release-date/lartpang/PyIRSTDMetrics"> </br> </div>

[!important]

Our exploration in this field continues with PyIRSTDMetrics, a project born from the same core motivation as our previous project PySODMetrics. Think of them as twin initiatives: PySODMetrics maps the landscape of current evaluation, while its sibling (PyIRSTDMetrics) takes the next step to expand upon and rethink it. We'd love your star! 🌟

News

2025-9-23: Our paper (Rethinking Evaluation of Infrared Small Target Detection) accepted by NeurIPS 2025 (D&B) has been released on the arXiv (2509.16888).

Introduction

A simple and efficient implementation for the IRSTD performance analysis.

Based on numpy、scikit-image and scipy.
Verification based on https://github.com/XinyiYing/BasicIRSTD
The code structure is simple and easy to extend
The code is lightweight and fast

Your improvements and suggestions are welcome.

Supported Metrics

| Metric | Sample-based | Whole-based | Related Class | Level | | ----------------------------- | ---------------------- | ----------- | -------------------------------------------------------------------------------------------- | ------ | | IoU | max,avg,adp,bin (nIoU) | bin (IoU) | CMMetrics+IOUHandler | pixel | | F1 | max,avg,adp,bin | bin | CMMetrics+FmeasureHandler | pixel | | Precision | max,avg,adp,bin | bin | CMMetrics+PrecisionHandler | pixel | | Recall | max,avg,adp,bin | bin | CMMetrics+RecallHandler | pixel | | TPR | max,avg,adp,bin | bin | CMMetrics+TPRHandler | pixel | | FPR | max,avg,adp,bin | bin | CMMetrics+FPRHandler | pixel | | Pd/Fa | | ✔ | ProbabilityDetectionAndFalseAlarmRate (like MatchingBasedMetrics+DistanceOnlyMatching) | target | | Pd/Fa | | ✔ | ShootingRuleBasedProbabilityDetectionAndFalseAlarmRate | target | | Pd/Fa | | ✔ | MatchingBasedMetrics+DistanceOnlyMatching/OPDCMatching | target | | hIoU | | ✔ | MatchingBasedMetrics+OPDCMatching | hybrid | | hIoU-based loc error analysis | | ✔ | HierarchicalIoUBasedErrorAnalysis | | | hIoU-based seg error analysis | | ✔ | HierarchicalIoUBasedErrorAnalysis | |

NOTE:

If you want to align the original implementation, use DistanceOnlyMatching/ProbabilityDetectionAndFalseAlarmRate.
If you want a more reasonable matching effect, use OPDCMatching we designed.
hIoU is a new metric that balances both pixel-level and target-level performance analysis and we provide a detailed error analysis tool based on it.

As shown in plot_average_metrics of examples/metric_recorder.py:

precision and recall sequences can be used to plot the PR curve.
TPR and FPR sequences can be used to plot the ROC curve.

Usage

The core files are in the folder py_irstd_metrics.

[Latest, but may be unstable] Install from the source code: pip install git+https://github.com/lartpang/PyIRSTDMetrics.git
[More stable] Install from PyPI: pip install pyirstdmetrics

Examples

Performance Improvements

In v1.0.2, we significantly optimized the OPDC matching computation (OPDCMatching).

Benchmark: v1.0.1 vs v1.0.2

To reproduce the benchmark, see examples/benchmark_metrics.py.

Thanks

BasicIRSTD is a PyTorch-based open-source and easy-to-use toolbox for infrared small target detction (IRSTD).

Citation

@misc{HierarchicalIoUBasedAnalysis,
  title         = {Rethinking Evaluation of Infrared Small Target Detection},
  author        = {Youwei Pang and Xiaoqi Zhao and Lihe Zhang and Huchuan Lu and Georges El Fakhri and Xiaofeng Liu and Shijian Lu},
  year          = {2025},
  eprint        = {2509.16888},
  archiveprefix = {arXiv},
  primaryclass  = {cs.CV},
}
@inproceedings{IRSTD-ACM-nIoU,
  title     = {Asymmetric Contextual Modulation for Infrared Small Target Detection},
  booktitle = WACV,
  author    = {Dai, Yimian and Wu, Yiquan and Zhou, Fei and Barnard, Kobus},
  year      = {2021},
  volume    = {},
  number    = {},
  pages     = {949-958},
  doi       = {10.1109/WACV48630.2021.00099},
  issn      = {2642-9381},
  month     = {Jan},
}
@article{IRSTD-DNANet-PdFa,
  title   = {Dense Nested Attention Network for Infrared Small Target Detection},
  author  = {Li, Boyang and Xiao, Chao and Wang, Longguang and Wang, Yingqian and Lin, Zaiping and Li, Miao and An, Wei and Guo, Yulan},
  journal = IEEE_J_IP,
  year    = {2023},
  volume  = {32},
  number  = {},
  pages   = {1745-1758},
  doi     = {10.1109/TIP.2022.3199107},
  issn    = {1941-0042},
  month   = {},
}
@article{MIRSTD-ShootingRules-DTUM,
  title   = {Direction-Coded Temporal U-Shape Module for Multiframe Infrared Small Target Detection},
  author  = {Ruojing Li and Wei An and Chao Xiao and Boyang Li and Yingqian Wang and Miao Li and Yulan Guo},
  journal = {IEEE Transactions on Neural Networks and Learning Systems},
  year    = {2023},
  volume  = {36},
  pages   = {555-568},
}

PyIRSTDMetrics

Install / Use

README