RT DETR

English | 简体中文

<h2 align="center">RT-DETR: DETRs Beat YOLOs on Real-time Object Detection</h2> <p align="center"> <!-- <a href="https://github.com/lyuwenyu/RT-DETR/blob/main/LICENSE"> <img alt="license" src="https://img.shields.io/badge/LICENSE-Apache%202.0-blue"> </a> --> <a href="https://github.com/lyuwenyu/RT-DETR/blob/main/LICENSE"> <img alt="license" src="https://img.shields.io/github/license/lyuwenyu/RT-DETR"> </a> <a href="https://github.com/lyuwenyu/RT-DETR/pulls"> <img alt="prs" src="https://img.shields.io/github/issues-pr/lyuwenyu/RT-DETR"> </a> <a href="https://github.com/lyuwenyu/RT-DETR/issues"> <img alt="issues" src="https://img.shields.io/github/issues/lyuwenyu/RT-DETR?color=pink"> </a> <a href="https://github.com/lyuwenyu/RT-DETR"> <img alt="issues" src="https://img.shields.io/github/stars/lyuwenyu/RT-DETR"> </a> <a href="https://arxiv.org/abs/2304.08069"> <img alt="arXiv" src="https://img.shields.io/badge/arXiv-2304.08069-red"> </a> <a href="mailto: lyuwenyu@foxmail.com"> <img alt="emal" src="https://img.shields.io/badge/contact_me-email-yellow"> </a> </p>

---

This is the official implementation of papers

• DETRs Beat YOLOs on Real-time Object Detection
• RT-DETRv2: Improved Baseline with Bag-of-Freebies for Real-Time Detection Transformer

<details> <summary>Fig</summary> <table><tr> <td><img src=https://github.com/lyuwenyu/RT-DETR/assets/77494834/0ede1dc1-a854-43b6-9986-cf9090f11a61 border=0 width=500></td> <td><img src=https://github.com/user-attachments/assets/437877e9-1d4f-4d30-85e8-aafacfa0ec56 border=0 width=500></td> </tr></table> </details>

🚀 Updates

• [2025.11.18] Release the newest member of the RT-DETR family: RT-DETRv4:Painlessly Furthering Real-Time Object Detection with Vision Foundation Models. By harnessing the rapidly evolving capabilities of Vision Foundation Models (VFMs), we boost lightweight detectors and, without incurring any extra inference latency, significantly improve the performance of the full-size model.
• [2024.11.28] Add torch tool for parameters and flops statistics. see run_profile.py
• [2024.10.10] Add sliced inference support for small object detecion. #468
• [2024.09.23] Add ✅Regnet and DLA34 for RTDETR.
• [2024.08.27] Add hubconf.py file to support torch hub.
• [2024.08.22] Improve the performance of ✅ RT-DETRv2-S to 48.1 mAP (<font color=green>+1.6</font> compared to RT-DETR-R18).
• [2024.07.24] Release ✅ RT-DETRv2!
• [2024.02.27] Our work has been accepted to CVPR 2024!
• [2024.01.23] Fix difference on data augmentation with paper in rtdetr_pytorch #84.
• [2023.11.07] Add pytorch ✅ rtdetr_r34vd for requests #107, #114.
• [2023.11.05] Upgrade the logic of remap_mscoco_category to facilitate training of custom datasets, see detils in Train custom data part. #81.
• [2023.10.23] Add discussion for deployments, supported onnxruntime, TensorRT, openVINO.
• [2023.10.12] Add tuning code for pytorch version, now you can tuning rtdetr based on pretrained weights.
• [2023.09.19] Upload ✅ pytorch weights convert from paddle version.
• [2023.08.24] Release RT-DETR-R18 pretrained models on objects365. 49.2 mAP and 217 FPS.
• [2023.08.22] Upload ✅ rtdetr_pytorch source code. Please enjoy it!
• [2023.08.15] Release RT-DETR-R101 pretrained models on objects365. 56.2 mAP and 74 FPS.
• [2023.07.30] Release RT-DETR-R50 pretrained models on objects365. 55.3 mAP and 108 FPS.
• [2023.07.28] Fix some bugs, and add some comments. 1, 2.
• [2023.07.13] Upload ✅ training logs on coco.
• [2023.05.17] Release RT-DETR-R18, RT-DETR-R34, RT-DETR-R50-m（example for scaled).
• [2023.04.17] Release RT-DETR-R50, RT-DETR-R101, RT-DETR-L, RT-DETR-X.

📣 News

• RTDETR and RTDETRv2 are now available in Hugging Face Transformers. #413, #549
• RTDETR is now available in ultralytics/ultralytics.

📍 Implementations

• 🔥 RT-DETRv2
  • paddle: code&weight
  • pytorch: code&weight
• 🔥 RT-DETR
  • paddle: code&weight
  • pytorch: code&weight

| Model | Input shape | Dataset | $AP^{val}$ | $AP^{val}_{50}$| Params(M) | FLOPs(G) | T4 TensorRT FP16(FPS) |:---:|:---:| :---:|:---:|:---:|:---:|:---:|:---:| | RT-DETR-R18 | 640 | COCO | 46.5 | 63.8 | 20 | 60 | 217 | | RT-DETR-R34 | 640 | COCO | 48.9 | 66.8 | 31 | 92 | 161 | | RT-DETR-R50-m | 640 | COCO | 51.3 | 69.6 | 36 | 100 | 145 | | RT-DETR-R50 | 640 | COCO | 53.1 | 71.3 | 42 | 136 | 108 | | RT-DETR-R101 | 640 | COCO | 54.3 | 72.7 | 76 | 259 | 74 | | RT-DETR-HGNetv2-L | 640 | COCO | 53.0 | 71.6 | 32 | 110 | 114 | | RT-DETR-HGNetv2-X | 640 | COCO | 54.8 | 73.1 | 67 | 234 | 74 | | RT-DETR-R18 | 640 | COCO + Objects365 | 49.2 | 66.6 | 20 | 60 | 217 | | RT-DETR-R50 | 640 | COCO + Objects365 | 55.3 | 73.4 | 42 | 136 | 108 | | RT-DETR-R101 | 640 | COCO + Objects365 | 56.2 | 74.6 | 76 | 259 | 74 | RT-DETRv2-S | 640 | COCO | 48.1 <font color=green>(+1.6)</font> | 65.1 | 20 | 60 | 217 | RT-DETRv2-M<sup>*<sup> | 640 | COCO | 49.9 <font color=green>(+1.0)</font> | 67.5 | 31 | 92 | 161 | RT-DETRv2-M | 640 | COCO | 51.9 <font color=green>(+0.6)</font> | 69.9 | 36 | 100 | 145 | RT-DETRv2-L | 640 | COCO | 53.4 <font color=green>(+0.3)</font> | 71.6 | 42 | 136 | 108 | RT-DETRv2-X | 640 | COCO | 54.3 | 72.8 <font color=green>(+0.1)</font> | 76 | 259| 74 |

Notes:

• COCO + Objects365 in the table means finetuned model on COCO using pretrained weights trained on Objects365.

🦄 Performance

🏕️ Complex Scenarios

<div align="center"> <img src="https://github.com/lyuwenyu/RT-DETR/assets/77494834/52743892-68c8-4e53-b782-9f89221739e4" width=500 > </div>

🌋 Difficult Conditions

<div align="center"> <img src="https://github.com/lyuwenyu/RT-DETR/assets/77494834/213cf795-6da6-4261-8549-11947292d3cb" width=500 > </div>

Citation

If you use RT-DETR or RTDETRv2 in your work, please use the following BibTeX entries:

@misc{lv2023detrs,
      title={DETRs Beat YOLOs on Real-time Object Detection},
      author={Yian Zhao and Wenyu Lv and Shangliang Xu and Jinman Wei and Guanzhong Wang and Qingqing Dang and Yi Liu and Jie Chen},
      year={2023},
      eprint={2304.08069},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

@misc{lv2024rtdetrv2improvedbaselinebagoffreebies,
      title={RT-DETRv2: Improved Baseline with Bag-of-Freebies for Real-Time Detection Transformer}, 
      author={Wenyu Lv and Yian Zhao and Qinyao Chang and Kui Huang and Guanzhong Wang and Yi Liu},
      year={2024},
      eprint={2407.17140},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2407.17140}, 
}