Onediff

<p align="center"> <img src="imgs/onediff_logo.png" height="100"> </p> <p align="center"> <a href="https://pypi.org/project/onediff" target="_blank"><img src="https://img.shields.io/pypi/v/onediff"></a> <a href="https://pypistats.org/packages/onediff" target="_blank"><img src="https://img.shields.io/pypi/dm/onediff?style=square"></a> <a href="https://github.com/siliconflow/onediff?tab=Apache-2.0-1-ov-file#readme" target="_blank"><img src="https://img.shields.io/github/license/siliconflow/onediff"></a> <a href="https://github.com/siliconflow/onediff/issues?q=is%3Aissue+is%3Aclosed" target="_blank"><img src="https://img.shields.io/github/issues-closed/siliconflow/onediff?color=blue"></a> <a href="https://github.com/siliconflow/onediff/issues?q=is%3Aopen+is%3Aissue" target="_blank"><img src="https://img.shields.io/github/issues/siliconflow/onediff"></a> </p> <p align="center"> <a href="https://github.com/siliconflow/onediff/stargazers" target="_blank"><img src="https://img.shields.io/github/stars/siliconflow/onediff?style=square&label=Stars&color=green"></a> <a href="https://twitter.com/search?q=%22onediff%22&src=typed_query&f=live" target="_blank"><img src="https://img.shields.io/badge/Twitter-Discuss-green?logo=twitter&amp"></a> <a href="https://github.com/siliconflow/onediff/actions/workflows/sd.yml" target="_blank"><img src="https://github.com/siliconflow/onediff/actions/workflows/sd.yml/badge.svg"></a> <a href="https://github.com/siliconflow/onediff/actions/workflows/examples.yml?query=event%3Aschedule" target="_blank"><img src="https://github.com/siliconflow/onediff/actions/workflows/examples.yml/badge.svg?event=schedule"></a> </p> <p align="center"> | <a href="https://github.com/siliconflow/onediff?tab=readme-ov-file#documentation"><b>Documentation</b></a> | <a href="https://github.com/siliconflow/onediff/wiki"><b>Community</b></a> | <a href="https://github.com/siliconflow/onediff/wiki/Contribution-Guide"><b>Contribution</b></a> | <a href="https://discord.gg/RKJTjZMcPQ"><b>Discord</b></a> | </p>

---

onediff is an out-of-the-box acceleration library for diffusion models, it provides:

• Out-of-the-box acceleration for popular UIs/libs(such as HF diffusers and ComfyUI)
• PyTorch code compilation tools and strong optimized GPU Kernels for diffusion models

News

• [2024/07/23] :rocket: Up to 1.7x Speedup for Kolors: Kolors Acceleration Report
• [2024/06/18] :rocket: Acceleration for DiT models: SD3 Acceleration Report, PixArt Acceleration Report, and Latte Acceleration Report
• [2024/04/13] :rocket: OneDiff 1.0 is released (Acceleration of SD & SVD with one line of code)
• [2024/01/12] :rocket: Accelerating Stable Video Diffusion 3x faster with OneDiff DeepCache + Int8
• [2023/12/19] :rocket: Accelerating SDXL 3x faster with DeepCache and OneDiff

Hiring

We're hiring! If you are interested in working on onediff at SiliconFlow, we have roles open for Interns and Engineers in Beijing (near Tsinghua University).

If you have contributed significantly to open-source software and are interested in remote work, you can contact us at talent@siliconflow.cn with onediff in the email title.

---

<!-- toc -->

• Documentation
  • Use with HF diffusers and ComfyUI
  • Performance comparison
    • SDXL E2E time
    • SVD E2E time
  • Quality Evaluation
  • Community and Support
  • Installation
    • 0. OS and GPU Compatibility
    • 1. Install torch and diffusers
    • 2. Install a compiler backend
      • Nexfort
      • OneFlow
    • 3. Install onediff
• More about onediff
  • Architecture
  • Features
  • Acceleration for State-of-the-art models
  • Acceleration for production environment
    • PyTorch Module compilation
    • Avoid compilation time for new input shape
    • Avoid compilation time for online serving
    • Distributed Run
  • OneDiff Enterprise Solution

<!-- tocstop -->

Documentation

onediff is the abbreviation of "one line of code to accelerate diffusion models".

Use with HF diffusers and ComfyUI

• onediff for HF diffusers 🤗
• onediff for ComfyUI
• onediff for Stable Diffusion web UI

Performance comparison

<img src="imgs/replace_a100.png" height="400">

SDXL E2E time

• Model stabilityai/stable-diffusion-xl-base-1.0;
• Image size 1024*1024, batch size 1, steps 30;
• NVIDIA A100 80G SXM4;

<img src="imgs/0_12_sdxl.png" height="400">

SVD E2E time

• Model stabilityai/stable-video-diffusion-img2vid-xt;
• Image size 576*1024, batch size 1, steps 25, decoder chunk size 5;
• NVIDIA A100 80G SXM4;

<img src="imgs/0_12_svd.png" height="400">

Note that we haven't got a way to run SVD with TensorRT on Feb 29 2024.

Quality Evaluation

We also maintain a repository for benchmarking the quality of generation after acceleration: odeval

Community and Support

• Create an issue
• Chat in Discord:
• Community and Feedback

Installation

0. OS and GPU Compatibility

• Linux
  • If you want to use onediff on Windows, please use it under WSL.
  • The guide to install onediff in WSL2.
• NVIDIA GPUs
  • Compatibility with Nvidia GPUs.

1. Install torch and diffusers

Note: You can choose the latest versions you want for diffusers or transformers.

python3 -m pip install "torch" "transformers==4.27.1" "diffusers[torch]==0.19.3"

2. Install a compiler backend

When considering the choice between OneFlow and Nexfort, either one is optional, and only one is needed.

• For DiT structural models or H100 devices, it is recommended to use Nexfort.

• For all other cases, it is recommended to use OneFlow. Note that optimizations within OneFlow will gradually transition to Nexfort in the future.

Nexfort

Install Nexfort is Optional. The detailed introduction of Nexfort is here.

python3 -m  pip install -U torch==2.3.0 torchvision==0.18.0 torchaudio==2.3.0 torchao==0.1
python3 -m  pip install -U nexfort

OneFlow

Install OneFlow is Optional.

NOTE: We have updated OneFlow frequently for onediff, so please install OneFlow by the links below.

• CUDA 11.8

  For NA/EU users

  python3 -m pip install -U --pre oneflow -f https://github.com/siliconflow/oneflow_releases/releases/expanded_assets/community_cu118
  

  For CN users

  python3 -m pip install -U --pre oneflow -f https://oneflow-pro.oss-cn-beijing.aliyuncs.com/branch/community/cu118
  

<details> <summary> Click to get OneFlow packages for other CUDA versions. </summary>

• CUDA 12.1

  For NA/EU users

  python3 -m pip install -U --pre oneflow -f https://github.com/siliconflow/oneflow_releases/releases/expanded_assets/community_cu122
  

  For CN users

  python3 -m pip install -U --pre oneflow -f https://oneflow-pro.oss-cn-beijing.aliyuncs.com/branch/community/cu122
  

• CUDA 12.2

  For NA/EU users

  python3 -m pip install -U --pre oneflow -f https://github.com/siliconflow/oneflow_releases/releases/expanded_assets/community_cu122
  

  For CN users

  python3 -m pip install -U --pre oneflow -f https://oneflow-pro.oss-cn-beijing.aliyuncs.com/branch/community/cu122
  

</details>

3. Install onediff

• From PyPI

python3 -m pip install --pre onediff

• From source

git clone https://github.com/siliconflow/onediff.git

cd onediff && python3 -m pip install -e .

Or install for development:

# install for dev
cd onediff && python3 -m pip install -e '.[dev]'

# code formatting and linting
pip3 install pre-commit
pre-commit install
pre-commit run --all-files

NOTE: If you intend to utilize plugins for ComfyUI/StableDiffusion-WebUI, we highly recommend installing OneDiff from the source rather than PyPI. This is necessary as you