[⬇️ Download] [📖 Website] [☸ Quick Start (Kubernetes)] [🤓 Quick Start (nerdctl)] [❓ FAQs & Troubleshooting]

Nydus: Dragonfly Container Image Service

<p><img src="misc/logo.svg" width="170"></p>

Introduction

Nydus implements a content-addressable file system on the RAFS format, which enhances the current OCI image specification by improving container launch speed, image space and network bandwidth efficiency, and data integrity.

The following Benchmarking results demonstrate that Nydus images significantly outperform OCI images in terms of container cold startup elapsed time on Containerd, particularly as the OCI image size increases.

Principles

Provide Fast, Secure And Easy Access to Data Distribution

• Performance: Second-level container startup speed, millisecond-level function computation code package loading speed.
• Low Cost: Written in memory-safed language Rust, numerous optimizations help improve memory, CPU, and network consumption.
• Flexible: Supports container runtimes such as runC and Kata, and provides Confidential Containers and vulnerability scanning capabilities
• Security: End to end data integrity check, Supply Chain Attack can be detected and avoided at runtime.

Key features

• On-demand Load: Container images/packages are downloaded on-demand in chunk unit to boost startup.
• Chunk Deduplication: Chunk level data de-duplication cross-layer or cross-image to reduce storage, transport, and memory cost.
• Compatible with Ecosystem: Storage backend support with Registry, OSS, NAS, Shared Disk, and P2P service. Compatible with the OCI images, and provide native eStargz images support.
• Data Analyzability: Record accesses, data layout optimization, prefetch, IO amplification, abnormal behavior detection.
• POSIX Compatibility: In-Kernel EROFS or FUSE filesystems together with overlayfs provide full POSIX compatibility
• I/O optimization: Use merged filesystem tree, data prefetching and User I/O amplification to reduce read latency and improve user I/O performance.

Ecosystem

Nydus tools

| Tool | Description | | -------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | | nydusd | Nydus user-space daemon, it processes all fscache/FUSE messages from the kernel and parses Nydus images to fullfil those requests | | nydus-image | Convert a single layer of OCI format container image into a nydus format container image generating meta part file and data part file respectively | | nydusify | It pulls OCI image down and unpack it, invokes nydus-image create to convert image and then pushes the converted image back to registry and data storage | | nydusctl | Nydusd CLI client (nydus-image inspect), query daemon's working status/metrics and configure it | | nydus-docker-graphdriver | [Experimental] Works as a docker remote graph driver to control how images and containers are stored and managed | | nydus-overlayfs | Containerd mount helper to invoke overlayfs mount with tweaking mount options a bit. So nydus prerequisites can be passed to vm-based runtime | | nydus-backend-proxy | A simple HTTP server to serve local directory as a blob backend for nydusd |

Supported platforms

| Type | Platform | Description | Status | | ------------- | --------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------ | | Storage | Registry/OSS/S3/NAS | Support for OCI-compatible distribution implementations such as Docker Hub, Harbor, Github GHCR, Aliyun ACR, NAS, and Aliyun OSS-like object storage service | ✅ | | Storage/Build | Harbor | Provides a general service for Harbor to support acceleration image conversion based on kinds of accelerator like Nydus and eStargz etc | ✅ | | Distribution | Dragonfly | Improve the runtime performance of Nydus image even further with the Dragonfly P2P data distribution system | ✅ | | Build | Buildkit | Provides the ability to build and export Nydus images directly from Dockerfile | ✅ | | Build/Runtime | Nerdctl | The containerd client to build or run (requires nydus snapshotter) Nydus image | ✅ | | Runtime | Docker / Moby | Run Nydus image in Docker container with containerd and nydus-snapshotter | ✅ | | Runtime | Kubernetes | Run Nydus image using CRI interface | ✅ | | Runtime | Containerd | Nydus Snapshotter, a containerd remote plugin to run Nydus image | ✅ | | Runtime | CRI-O / Podman | Run Nydus image with CRI-O or Podman | 🚧 | | Runtime | [KataContainers](https://github.com/kata-containers/kata-containers/blob/main