Mmdeploy

<div align="center"> <img src="resources/mmdeploy-logo.png" width="450"/> <div>&nbsp;</div> <div align="center"> <b><font size="5">OpenMMLab website</font></b> ^{<a href="https://openmmlab.com"> <i><font size="4">HOT</font></i> </a>} &nbsp;&nbsp;&nbsp;&nbsp; <b><font size="5">OpenMMLab platform</font></b> ^{<a href="https://platform.openmmlab.com"> <i><font size="4">TRY IT OUT</font></i> </a>} </div> <div>&nbsp;</div>

English | 简体中文

</div> <div align="center"> <a href="https://openmmlab.medium.com/" style="text-decoration:none;"> <img src="https://user-images.githubusercontent.com/25839884/218352562-cdded397-b0f3-4ca1-b8dd-a60df8dca75b.png" width="3%" alt="" /></a> <img src="https://user-images.githubusercontent.com/25839884/218346358-56cc8e2f-a2b8-487f-9088-32480cceabcf.png" width="3%" alt="" /> <a href="https://discord.gg/raweFPmdzG" style="text-decoration:none;"> <img src="https://user-images.githubusercontent.com/25839884/218347213-c080267f-cbb6-443e-8532-8e1ed9a58ea9.png" width="3%" alt="" /></a> <img src="https://user-images.githubusercontent.com/25839884/218346358-56cc8e2f-a2b8-487f-9088-32480cceabcf.png" width="3%" alt="" /> <a href="https://twitter.com/OpenMMLab" style="text-decoration:none;"> <img src="https://user-images.githubusercontent.com/25839884/218346637-d30c8a0f-3eba-4699-8131-512fb06d46db.png" width="3%" alt="" /></a> <img src="https://user-images.githubusercontent.com/25839884/218346358-56cc8e2f-a2b8-487f-9088-32480cceabcf.png" width="3%" alt="" /> <a href="https://www.youtube.com/openmmlab" style="text-decoration:none;"> <img src="https://user-images.githubusercontent.com/25839884/218346691-ceb2116a-465a-40af-8424-9f30d2348ca9.png" width="3%" alt="" /></a> </div>

Highlights

The MMDeploy 1.x has been released, which is adapted to upstream codebases from OpenMMLab 2.0. Please align the version when using it. The default branch has been switched to main from master. MMDeploy 0.x (master) will be deprecated and new features will only be added to MMDeploy 1.x (main) in future.

| mmdeploy | mmengine | mmcv | mmdet | others | | :------: | :------: | :------: | :------: | :----: | | 0.x.y | - | <=1.x.y | <=2.x.y | 0.x.y | | 1.x.y | 0.x.y | 2.x.y | 3.x.y | 1.x.y |

deploee offers over 2,300 AI models in ONNX, NCNN, TRT and OpenVINO formats. Featuring a built-in list of real hardware devices, deploee enables users to convert Torch models into any target inference format for profiling purposes.

Introduction

MMDeploy is an open-source deep learning model deployment toolset. It is a part of the OpenMMLab project.

<div align="center"> <img src="resources/introduction.png"> </div>

Main features

Fully support OpenMMLab models

The currently supported codebases and models are as follows, and more will be included in the future

• mmpretrain
• mmdet
• mmseg
• mmagic
• mmocr
• mmpose
• mmdet3d
• mmrotate
• mmaction2

Multiple inference backends are available

The supported Device-Platform-InferenceBackend matrix is presented as following, and more will be compatible.

The benchmark can be found from here

<div style="width: fit-content; margin: auto;"> <table> <tr> <th>Device / <br> Platform</th> <th>Linux</th> <th>Windows</th> <th>macOS</th> <th>Android</th> </tr> <tr> <th>x86_64 <br> CPU</th> <td> _{<a href="https://github.com/open-mmlab/mmdeploy/actions/workflows/backend-ort.yml"><img src="https://img.shields.io/github/actions/workflow/status/open-mmlab/mmdeploy/backend-ort.yml"></a>} _onnxruntime <br> _{<a href="https://github.com/open-mmlab/mmdeploy/actions/workflows/backend-pplnn.yml"><img src="https://img.shields.io/github/actions/workflow/status/open-mmlab/mmdeploy/backend-pplnn.yml"></a>} _pplnn <br> _{<a href="https://github.com/open-mmlab/mmdeploy/actions/workflows/backend-ncnn.yml"><img src="https://img.shields.io/github/actions/workflow/status/open-mmlab/mmdeploy/backend-ncnn.yml"></a>} _ncnn <br> _{<a href="https://github.com/open-mmlab/mmdeploy/actions/workflows/backend-torchscript.yml"><img src="https://img.shields.io/github/actions/workflow/status/open-mmlab/mmdeploy/backend-torchscript.yml"></a>} _LibTorch <br> _{<img src="https://img.shields.io/badge/build-no%20status-lightgrey">} _OpenVINO <br> _{<img src="https://img.shields.io/badge/build-no%20status-lightgrey">} _TVM <br> </td> <td> _{<img src="https://img.shields.io/badge/build-no%20status-lightgrey">} _onnxruntime <br> _{<img src="https://img.shields.io/badge/build-no%20status-lightgrey">} _OpenVINO <br> _{<img src="https://img.shields.io/badge/build-no%20status-lightgrey">} _ncnn <br> </td> <td align="center"> - </td> <td align="center"> - </td> </tr> <tr> <th>ARM <br> CPU</th> <td> _{<a href="https://github.com/open-mmlab/mmdeploy/actions/workflows/build.yml"><img src="https://byob.yarr.is/open-mmlab/mmdeploy/cross_build_aarch64"></a>} _ncnn <br> </td> <td align="center"> - </td> <td align="center"> - </td> <td align="center"> _{<a href="https://github.com/open-mmlab/mmdeploy/actions/workflows/backend-ncnn.yml"><img src="https://img.shields.io/github/actions/workflow/status/open-mmlab/mmdeploy/backend-ncnn.yml"></a>} _ncnn <br> </td> </tr> <tr> <th>RISC-V</th> <td> _{<a href="https://github.com/open-mmlab/mmdeploy/actions/workflows/linux-riscv64-gcc.yml"><img src="https://img.shields.io/github/actions/workflow/status/open-mmlab/mmdeploy/linux-riscv64-gcc.yml"></a>} _ncnn <br> </td> <td align="center"> - </td> <td align="center"> - </td> <td align="center"> - </td> </tr> <tr> <th>NVIDIA <br> GPU</th> <td> _{<a href="https://github.com/open-mmlab/mmdeploy/actions/workflows/build.yml"><img src="https://byob.yarr.is/open-mmlab/mmdeploy/build_cuda113_linux"></a>} _onnxruntime <br> _{<a href="https://github.com/open-mmlab/mmdeploy/actions/workflows/build.yml"><img src="https://byob.yarr.is/open-mmlab/mmdeploy/build_cuda113_linux"></a>} _TensorRT <br> _{<img src="https://img.shields.io/badge/build-no%20status-lightgrey">} _LibTorch <br> _{<a href="https://github.com/open-mmlab/mmdeploy/actions/workflows/backend-pplnn.yml"><img src="https://img.shields.io/github/actions/workflow/status/open-mmlab/mmdeploy/backend-pplnn.yml"></a>} _pplnn <br> </td> <td> _{<a href="https://github.com/open-mmlab/mmdeploy/actions/workflows/build.yml"><img src="https://byob.yarr.is/open-mmlab/mmdeploy/build_cuda113_windows"></a>} _onnxruntime <br> _{<a href="https://github.com/open-mmlab/mmdeploy/actions/workflows/build.yml"><img src="https://byob.yarr.is/open-mmlab/mmdeploy/build_cuda113_windows"></a>} _TensorRT <br> </td> <td align="center"> - </td> <td align="center"> - </td> </tr> <tr> <th>NVIDIA <br> Jetson</th> <td> _{<img src="https://img.shields.io/badge/build-no%20status-lightgrey">} _TensorRT <br> </td> <td align="center"> - </td> <td align="center"> - </td> <td align="center"> - </td> </tr> <tr> <th>Huawei <br> ascend310</th> <td> _{<a href="https://github.com/open-mmlab/mmdeploy/actions/workflows/backend-ascend.yml"><img src="https://img.shields.io/github/actions/workflow/status/open-mmlab/mmdeploy/backend-ascend.yml"></a>} _CANN <br> </td> <td align="center"> - </td> <td align="center"> - </td> <td align="center"> - </td> </tr> <tr> <th>Rockchip</th> <td> _{<a href="https://github.com/open-mmlab/mmdeploy/actions/workflows/backend-rknn.yml"><img src="https://img.shields.io/github/actions/workflow/status/open-mmlab/mmdeploy/backend-rknn.yml"></a>} _RKNN <br> </td> <td align="center"> - </td> <td align="center"> - </td> <td align="center"> - </td> </tr> <tr> <th>Apple M1</th> <td align="center"> - </td>