Usls

A Rust library integrated with ONNXRuntime, providing a collection of Computer Vison and Vision-Language models such as YOLO, FastVLM, and more.

Generate Convert Improve

Install / Use

/learn @jamjamjon/Usls

About this skill

Quality Score

0/100

README

usls is a cross-platform Rust library powered by ONNX Runtime for efficient inference of SOTA vision and vision-language models (typically under 1B parameters).

<img src='https://github.com/jamjamjon/assets/releases/download/images/pipeline.png'> (Generated by Seedream4.5)

🌟 Highlights

⚡ High Performance: Multi-threading, SIMD, and CUDA-accelerated processing
✨ Cross-Platform: Linux, macOS, Windows with ONNX Runtime execution providers (CUDA, TensorRT, CoreML, OpenVINO, DirectML, etc.)
🎯 Precision Support: FP32, FP16, INT8, UINT8, Q4, Q4F16, BNB4, and more
🛠️ Full-Stack Suite: DataLoader, Annotator, and Viewer for complete workflows
🏗️ Unified API: Single Model trait inference with run()/forward()/encode_images()/encode_texts() and unified Y output
📥 Auto-Management: Automatic model download (HuggingFace/GitHub), caching and path resolution
📦 Multiple Inputs: Image, directory, video, webcam, stream and combinations
🌱 Model Ecosystem: 50+ SOTA vision and VLM models

🚀 Quick Start

Run the YOLO-Series demo to explore models with different tasks, precision and execution providers:

Tasks: detect, segment, pose, classify, obb
Versions: v5, v6, v7, v8, v9, v10, 11, 12, v13, 26
Scales: n, s, m, l, x
Precision: fp32, fp16, q8, int8, q4, q4f16, bnb4, and more
Execution Providers: CPU, CUDA, TensorRT, TensorRT-RTX, CoreML, OpenVINO, and more

cargo run -r --example yolo -- --task detect --ver 26 --scale n --dtype fp16

</details> <details> <summary>Nvidia CUDA + CUDA Image Processor</summary>

cargo run -r -F cuda --example yolo -- --task segment --ver 11 --scale m --device cuda:0 --processor-device cuda:0

</details> <details> <summary>Nvidia TensorRT + CUDA Image Processor</summary>

cargo run -r -F tensorrt-full --example yolo -- --device tensorrt:0 --processor-device cuda:0

</details> <details> <summary>Nvidia TensorRT-RTX + CUDA Image Processor</summary>

cargo run -r -F nvrtx-full --example yolo -- --device nvrtx:0 --processor-device cuda:0

</details> <details> <summary>Apple Silicon CoreML</summary>

cargo run -r -F coreml --example yolo -- --device coreml

</details> <details> <summary>Intel OpenVINO (CPU/GPU/VPU)</summary>

cargo run -r -F openvino -F ort-load-dynamic --example yolo -- --device openvino:CPU

</details> <details> <summary>📊 Performance Benchmarks</summary>

Environment: NVIDIA RTX 3060Ti (TensorRT-10.11.0.33, CUDA 12.8, TensorRT-RTX-1.3.0.35) / Intel i5-12400F

Setup: YOLO26 Detection, COCO2017-val (5,000 images), 640x640, Conf thresholds: [0.35, 0.3, ..]

Results are for rough reference only.

| Scale | EP | Image Processor | DType | Batch | Preprocess | Inference | Postprocess | Total | | --- | --- | --- | --- | --- | --- | --- | --- | --- | | n | TensorRT | CUDA | FP16 | 1 | ~233µs | ~1.3ms | ~14µs | ~1.55ms | | n | TensorRT-RTX | CUDA | FP32 | 1 | ~233µs | ~2.0ms | ~10µs | ~2.24ms | | n | TensorRT-RTX | CUDA | FP16 | 1 | ❓ | ❓ | ❓ | ❓ | | n | CUDA | CUDA | FP32 | 1 | ~233µs | ~5.0ms | ~17µs | ~5.25ms | | n | CUDA | CUDA | FP16 | 1 | ~233µs | ~3.6ms | ~17µs | ~3.85ms | | n | CUDA | CPU | FP32 | 1 | ~800µs | ~6.5ms | ~14µs | ~7.31ms | | n | CUDA | CPU | FP16 | 1 | ~800µs | ~5.0ms | ~14µs | ~5.81ms | | n | CPU | CPU | FP32 | 1 | ~970µs | ~20.5ms | ~14µs | ~21.48ms | | n | CPU | CPU | FP16 | 1 | ~970µs | ~25.0ms | ~14µs | ~25.98ms | | n | TensorRT | CUDA | FP16 | 8 | ~1.2ms | ~6.0ms | ~55µs | ~7.26ms | | n | TensorRT | CPU | FP16 | 8 | ~18.0ms | ~25.5ms | ~55µs | ~43.56ms | | m | TensorRT | CUDA | FP16 | 1 | ~233µs | ~3.6ms | ~14µs | ~3.85ms | | m | TensorRT | CUDA | Int8 | 1 | ~233µs | ~2.6ms | ~14µs | ~2.84ms | | m | CUDA | CUDA | FP32 | 1 | ~233µs | ~16.1ms | ~17µs | ~16.35ms | | m | CUDA | CUDA | FP16 | 1 | ~233µs | ~8.8ms | ~17µs | ~9.05ms |

</details>

🔍 What's Next?

<a href="https://jamjamjon.github.io/usls/">📖 Online Documentation</a>
<a href="https://docs.rs/usls/latest/usls/">📚 API Reference</a>
<a href="https://jamjamjon.github.io/usls/model-zoo/overview/">📦 Model Zoo</a>
<a href="./examples/README.md">🚀 Examples</a>

📦 Model Zoo

[!NOTE]

Status: ✅ Supported | ❓ Unknown | ❌ Not Supported For Now

🔍 All ONNX models are available from the ONNX Models Repository

<details closed> <summary>🔥 YOLO-Series</summary>

| Model | Task / Description | Demo | Dynamic Batch | TensorRT | FP32 | FP16 | Q8 | Q4f16 | BNB4 | | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | | YOLOv5 | Image Classification Object Detection Instance Segmentation | demo | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | | YOLOv6 | Object Detection | demo | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | | YOLOv7 | Object Detection | demo | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | | YOLOv8 | Object Detection Instance Segmentation Image Classification Oriented Object Detection Keypoint Detection | demo | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | | YOLO11 | Object Detection Instance Segmentation Image Classification Oriented Object Detection Keypoint Detection | demo | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | | YOLOv9 | Object Detection | demo | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | | YOLOv10 | Object Detection | demo | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | | YOLOv12 | Image Classification Object Detection Instance Segmentation | demo | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | YOLOv13 | Object Detection | demo | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | YOLO26 | Object Detection Instance Segmentation Image Classification Oriented Object Detection Keypoint Detection | demo | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |

</details> <details closed> <summary>🏷️ Image Classification & Tagging</summary>

| Model | Task / Description | Demo | Dynamic Batch | TensorRT | FP32 | FP16 | Q8 | Q4f16 | BNB4 | | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | | BEiT | Image Classification | demo | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | | ConvNeXt | Image Classification | demo | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | | FastViT | Image Classification | demo | ✅ | ✅| ✅ | ✅| ❌ | ❌ | ❌ | | MobileOne | Image Classification | demo | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | | DeiT | Image Classification | demo | ✅ | ✅ | ✅ | ✅| ❌ | ❌ | ❌ | | RAM | Image Tagging | demo | ✅ | ❓| ✅ | ✅| ✅ | ✅ | ✅ | | RAM++ | Image Tagging | demo | ✅ | ❓ | ✅ | ✅ | ✅ | ✅ | ✅ |

</details> <details closed> <summary>🎯 Object Detection</summary>

| Model | Task / Description | Demo | Dynamic Batch | TensorRT | FP32 | FP16 | Q8 | Q4f16 | BNB4 | | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | | RT-DETRv1 | Object Detection | demo | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | RT-DETRv2 | Object Detection | demo | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | RT-DETRv4 | Object Detection | demo | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | RF-DETR | Object Detection | demo| ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | PP-PicoDet | Object Detection | demo | ❌ | ❓ | ✅ | ❌ | ❌ | ❌ | ❌ | | D-FINE | Object Detection | demo | ✅ | ❓

Related Skills

node-connect

334.1k

Diagnose OpenClaw node connection and pairing failures for Android, iOS, and macOS companion apps

frontend-design

82.1k

Create distinctive, production-grade frontend interfaces with high design quality. Use this skill when the user asks to build web components, pages, or applications. Generates creative, polished code that avoids generic AI aesthetics.

openai-whisper-api

334.1k

Transcribe audio via OpenAI Audio Transcriptions API (Whisper).

commit-push-pr

82.1k

Commit, push, and open a PR