Savant: High-Performance Computer Vision Framework For Data Center And Edge

GitHub release (with filter)

:star: Star us on GitHub — it motivates us a lot and helps the project become more visible to developers.

current-demos-page

Savant is an open-source, high-level framework for building real-time, streaming, highly efficient multimedia AI applications on the Nvidia stack. It helps to develop dynamic, fault-tolerant inference pipelines that utilize the best Nvidia approaches for data center and edge accelerators.

Savant is built on DeepStream and provides a high-level abstraction layer for building inference pipelines. It is designed to be easy to use, flexible, and scalable. It is a great choice for building both real-time or high-load computer vision and video analytics applications.

What Version To Use

Savant depends on Nvidia DeepStream and JetPack versions (Jetson). The following tables show the compatibility of Savant versions with DeepStream versions.

0.2.11 (Jetpack 5.1+) Production Release (Outdated)

| Requirements | Status | DeepStream | |-------------------------------------------------|--------|------------| | X86 Driver 525(Datacenter), 530+ Quadro/GeForce | Stable | 6.3 | | Jetson Xavier, Orin with JetPack 5.1.2 GA | Stable | 6.3 |

0.4.x Previous Production Release (Outdated)

This release contains stable features and is tested for production use.

0.5.x Current Production Release

This release is recommended for production use. The release works on dGPU (Turing, Volta, Ampere, Ada) and Jetson Orin (Nano/NX/AGX) hardware.

This release is based on DeepStream 7.0.

0.6.x Current Development

This release is NOT recommended for production use. The release ADDS support for Blackwell GPUs and DROPS support for Pascal GPUs. It supports Jetson Orin (Nano/NX/AGX) hardware.

This release is based on a customized DeepStream 7.1 with TensorRT 10.9.

| Requirements | Status | DeepStream | TensorRT | |-------------------------------------------------|--------|------------|----------| | X86 Driver 570.133.20+ | Stable | 7.1 | 10.9 | | Jetson Orin JetPack 6.2 | Stable | 7.1 | 10.9 |

Chat With Us

The best way to approach us is Discord. We are always happy to help you with any questions you may have.

Quick Links

Getting Started

First, take a look at the runtime configuration guide to configure the working environment.

The demo shows a pipeline featuring person detection, facial detection, tracking, facial blurring (OpenCV CUDA), and a real-time analytics dashboard:

git clone https://github.com/insight-platform/Savant.git
cd Savant/samples/peoplenet_detector
git lfs pull

# if x86
../../utils/check-environment-compatible && docker compose -f docker-compose.x86.yml up

# if Jetson
../../utils/check-environment-compatible && docker compose -f docker-compose.l4t.yml up

# open 'rtsp://127.0.0.1:554/stream/city-traffic' in your player
# or visit 'http://127.0.0.1:888/stream/city-traffic/' (LL-HLS)

# Ctrl+C to stop running the compose bundle

# to get back to project root
cd ../..

Who Would Be Interested in Savant

If your task is to implement high-performance production-ready computer vision and video analytics applications, Savant is for you.

With Savant, developers:

get the maximum performance on Nvidia equipment on edge and in the core;
decrease time to market when building dynamic pipelines with DeepStream technology but without low-level programming;
develop easily maintainable and testable applications with a well-established framework API;
build heterogeneous pipelines with different models and data sources;
build hybrid edge/datacenter applications with the same codebase;
monitor and trace the pipelines with OpenTelemetry and Prometheus;
implement on-demand and non-linear processing by utilizing Replay.

Runs On Nvidia Hardware

Savant components, processing video and computer vision, require Nvidia hardware. We support the following devices:

Jetson Xavier NX/AGX (0.2.x);
Jetson Orin Nano/NX/AGX (0.3.x and newer);
Nvidia Turing, Ampere, Ada, Hopper, Blackwell GPUs (0.2.x and newer).

Why We Developed Savant

We developed Savant give computer vision and video analytics engineers a ready-to-use stack for building real-life computer vision applications working at the edge and in the data center. Unlike other computer vision frameworks like PyTorch, TensorFlow, OpenVINO/DlStreamer, and DeepStream, Savant provides users with not only inference and image manipulation tools but also advanced architecture for building distributed edge/datacenter computer vision applications communicating over the network. Thus, Savant users focus on computer vision but do not reinvent the wheel, when developing their applications.

Savant is a very high-level framework hiding low-level internals from developers: computer vision pipelines consist of declarative (YAML) blocks with Python functions.

Features

Savant is packed with many features skyrocketing the development of high-performing computer vision applications.

🔧 All You Need for Building Real-Life Applications

Savant supports everything you need for developing advanced pipelines: detection, classification, segmentation, tracking, and custom pre- and post-processing for meta and images.

We have implemented samples demonstrating pipelines you can build with Savant. Visit the samples folder to learn more.

🚀 High Performance

Savant is designed to be fast: it works on top of DeepStream - the fastest SDK for video analytics. Even the heavyweight segmentation models can run in real-time on Savant. See the Performance Regression Tracking Dashboard for the latest performance results.

🌐 Works On Edge and Data Center Equipment

The framework supports running the pipelines on both Nvidia's edge devices (Jetson Family) and data center devices ( Tesla, Quadro, etc.) with minor or zero changes.

❤️ Cloud-Ready

Savant pipelines run in Docker containers. We provide images for x86+dGPU and Jetson hardware. Integrated OpenTelemetry and Prometheus support enable monitoring and tracing of the pipelines.

⚡ Low Latency and High Capacity Processing

Savant can be configured to execute a pipeline in real-time, skipping data when running out of capacity or in high capacity mode, which guarantees the processing of all the data, maximizing the utilization of the available resources.

🤝 Ready-To-Use API

A pipeline is a self-sufficient service communicating with the world via high-performance streaming API. Whether developers use provided adapters or Client SDK, both approaches use the API.

📁 Advanced Data Protocol

The framework universally uses a common protocol for both video and metadata delivery. The protocol is highly flexible, allowing video-related information alongside arbitrary structures useful for IoT and 3rd-party integrations.

⏱ OpenTelemetry Support

In Savant, you can precisely instrument pipelines with OpenTelemetry: a unified monitoring solution. You can use sampled or complete traces to balance the performance and precision. The traces can span from edge to core to business logic through network and storage because their propagation is supported by the Savant protocol.

📊 Prometheus Support

Savant pipelines can be instrumented with Prometheus: a popular monitoring solution. Prometheus is a great choice for monitoring the pipeline's performance and resource utilization.

🧰 Client SDK

We provide Python-based SDK to interact with Savant pipelines (ingest and receive data). It enables simple integration with 3rd-party services. Client SDK is integrated with OpenTelemetry providing programmatic access to the pipeline traces and logs.

🧘 Development Server

Software development fo

Savant

Install / Use

README