<div align="center"> <img src="docs/assets/images/DeepSpeed_light.svg#gh-light-mode-only" width="400px"> <img src="docs/assets/images/DeepSpeed_dark_transparent.svg#gh-dark-mode-only" width="400px"> </div>

Latest News

• [2025/12] DeepSpeed Core API updates: PyTorch-style backward and low-precision master states

• [2025/11] DeepSpeed ZeRO++ powers large-scale distillation training of LLMs for Recommendation Systems at LinkedIn

• [2025/10] We hosted the Ray x DeepSpeed Meetup at Anyscale. We shared our most recent work on SuperOffload, ZenFlow, Muon Optimizer Support, Arctic Long Sequence Training and DeepCompile. Please find the meetup slides here.

• [2025/10] SuperOffload: Unleashing the Power of Large-Scale LLM Training on Superchips

• [2025/10] Study of ZenFlow and ZeRO offload performance with DeepSpeed CPU core binding

• [2025/08] ZenFlow: Stall-Free Offloading Engine for LLM Training

• [2025/06] Arctic Long Sequence Training (ALST) with DeepSpeed: Scalable And Efficient Training For Multi-Million Token Sequences

• [2025/06] DeepNVMe: Affordable I/O scaling for Deep Learning Applications

<!-- NOTE: we must use html for news items otherwise links will be broken in the 'more news' section --> <details> <!-- NOTE: Maintain only three items in 'more news' section --> <summary>More news</summary> <ul> <li>[2025/04] <a href="https://github.com/deepspeedai/DeepSpeed/blob/master/blogs/deepcompile/README.md">DeepCompile: Unlocking Compiler Optimization for Distributed Training</a></li> <li>[2025/03] <a href="https://github.com/deepspeedai/DeepSpeed/blob/master/blogs/huggingface-tp/README.md">DeepSpeed AutoTP: Automatic Tensor Parallel Training of Hugging Face models</a></li> <li>[2024/12] <a href="https://github.com/deepspeedai/DeepSpeed/blob/master/blogs/ulysses-offload/README.md">Ulysses-Offload: Democratizing Long Context LLM Training</a></li> </ul> </details>

---

Extreme Speed and Scale for DL Training

DeepSpeed enabled the world's most powerful language models (at the time of this writing) such as MT-530B and BLOOM. DeepSpeed offers a confluence of system innovations, that has made large scale DL training effective, and efficient, greatly improved ease of use, and redefined the DL training landscape in terms of scale that is possible. These innovations include ZeRO, ZeRO-Infinity, 3D-Parallelism, Ulysses Sequence Parallelism, DeepSpeed-MoE, etc.

---

DeepSpeed Adoption

DeepSpeed was an important part of Microsoft’s AI at Scale initiative to enable next-generation AI capabilities at scale, where you can find more information here.

DeepSpeed has been used to train many different large-scale models, below is a list of several examples that we are aware of (if you'd like to include your model please submit a PR):

• Megatron-Turing NLG (530B)
• Jurassic-1 (178B)
• BLOOM (176B)
• GLM (130B)
• xTrimoPGLM (100B)
• YaLM (100B)
• GPT-NeoX (20B)
• AlexaTM (20B)
• Turing NLG (17B)
• METRO-LM (5.4B)

DeepSpeed has been integrated with several different popular open-source DL frameworks such as:

| | Documentation | | ---------------------------------------------------------------------------------------------- | -------------------------------------------- | <img src="docs/assets/images/transformers-light.png#gh-light-mode-only" width="250px"><img src="docs/assets/images/transformers-dark.png#gh-dark-mode-only" width="250px"> | Transformers with DeepSpeed | | <img src="docs/assets/images/accelerate-light.png#gh-light-mode-only" width="250px"><img src="docs/assets/images/accelerate-dark.png#gh-dark-mode-only" width="250px"> | Accelerate with DeepSpeed | | <img src="docs/assets/images/lightning-light.svg#gh-light-mode-only" width="200px"><img src="docs/assets/images/lightning-dark.svg#gh-dark-mode-only" width="200px"> | Lightning with DeepSpeed | | <img src="docs/assets/images/mosaicml.svg" width="200px"> | MosaicML with DeepSpeed | | <img src="docs/assets/images/determined.svg" width="225px"> | Determined with DeepSpeed | | <img src="https://user-images.githubusercontent.com/58739961/187154444-fce76639-ac8d-429b-9354-c6fac64b7ef8.jpg" width=150> | MMEngine with DeepSpeed |

---

Build Pipeline Status

| Description | Status | | ----------- | ------ | | NVIDIA | | | AMD | | | CPU | | | Intel Gaudi | | | Intel XPU | | | Integrations | | | Misc | [](htt