Before moving on, please consider giving us a GitHub star ⭐️. Thank you!

Other languages: 简体中文日本語 한국어 Português

Website • Docs • Quick Start • Community Discord • Dragonfly Forum • Join the Dragonfly Community

GitHub Discussions • GitHub Issues • Contributing • AI Agents Guide • Dragonfly Cloud

The world's most efficient in-memory data store

Dragonfly is an in-memory data store built for modern application workloads.

Fully compatible with Redis and Memcached APIs, Dragonfly requires no code changes to adopt. Compared to legacy in-memory datastores, Dragonfly delivers 25X more throughput, higher cache hit rates with lower tail latency, and can run on up to 80% less resources for the same sized workload.

Benchmarks
Quick start
Configuration
Roadmap and status
Design decisions
Background
Build from source

<a name="benchmarks"><a/>Benchmarks

We first compare Dragonfly with Redis on m5.large instance which is commonly used to run Redis due to its single-threaded architecture. The benchmark program runs from another load-test instance (c5n) in the same AZ using memtier_benchmark -c 20 --test-time 100 -t 4 -d 256 --distinct-client-seed

Dragonfly shows a comparable performance:

SETs (--ratio 1:0):

| Redis | DF | | -----------------------------------------|----------------------------------------| | QPS: 159K, P99.9: 1.16ms, P99: 0.82ms | QPS:173K, P99.9: 1.26ms, P99: 0.9ms | | | |

GETs (--ratio 0:1):

| Redis | DF | | ----------------------------------------|----------------------------------------| | QPS: 194K, P99.9: 0.8ms, P99: 0.65ms | QPS: 191K, P99.9: 0.95ms, P99: 0.8ms |

The benchmark above shows that the algorithmic layer inside DF that allows it to scale vertically does not take a large toll when running single-threaded.

However, if we take a bit stronger instance (m5.xlarge), the gap between DF and Redis starts growing. (memtier_benchmark -c 20 --test-time 100 -t 6 -d 256 --distinct-client-seed):

SETs (--ratio 1:0):

| Redis | DF | | ----------------------------------------|----------------------------------------| | QPS: 190K, P99.9: 2.45ms, P99: 0.97ms | QPS: 279K , P99.9: 1.95ms, P99: 1.48ms|

GETs (--ratio 0:1):

| Redis | DF | | ----------------------------------------|----------------------------------------| | QPS: 220K, P99.9: 0.98ms , P99: 0.8ms | QPS: 305K, P99.9: 1.03ms, P99: 0.87ms |

Dragonfly throughput capacity continues to grow with instance size, while single-threaded Redis is bottlenecked on CPU and reaches local maxima in terms of performance.

If we compare Dragonfly and Redis on the most network-capable instance c6gn.16xlarge, Dragonfly showed a 25X increase in throughput compared to Redis single process, crossing 3.8M QPS.

Dragonfly's 99th percentile latency metrics at its peak throughput:

| op | r6g | c6gn | c7g | |-------|-------|-------|-------| | set | 0.8ms | 1ms | 1ms | | get | 0.9ms | 0.9ms | 0.8ms | | setex | 0.9ms | 1.1ms | 1.3ms |

All benchmarks were performed using memtier_benchmark (see below) with number of threads tuned per server and instance type. memtier was run on a separate c6gn.16xlarge machine. We set the expiry time to 500 for the SETEX benchmark to ensure it would survive the end of the test.

  memtier_benchmark --ratio ... -t <threads> -c 30 -n 200000 --distinct-client-seed -d 256 \
     --expiry-range=...

In pipeline mode --pipeline=30, Dragonfly reaches 10M QPS for SET and 15M QPS for GET operations.

Dragonfly vs. Memcached

We compared Dragonfly with Memcached on a c6gn.16xlarge instance on AWS.

With a comparable latency, Dragonfly throughput outperformed Memcached throughput in both write and read workloads. Dragonfly demonstrated better latency in write workloads due to contention on the write path in Memcached.

SET benchmark

| Server | QPS(thousands qps) | latency 99% | 99.9% | |:---------:|:------------------:|:-----------:|:-------:| | Dragonfly | 🟩 3844 |🟩 0.9ms | 🟩 2.4ms | | Memcached | 806 | 1.6ms | 3.2ms |

GET benchmark

| Server | QPS(thousands qps) | latency 99% | 99.9% | |-----------|:------------------:|:-----------:|:-------:| | Dragonfly | 🟩 3717 | 1ms | 2.4ms | | Memcached | 2100 | 🟩 0.34ms | 🟩 0.6ms |

Memcached exhibited lower latency for the read benchmark, but also lower throughput.

Memory efficiency

To test memory efficiency, we filled Dragonfly and Redis with ~5GB of data using the debug populate 5000000 key 1024 command, sent update traffic with memtier, and kicked off the snapshotting with the bgsave command.

This figure demonstrates how each server behaved in terms of memory efficiency.

Dragonfly was 30% more memory efficient than Redis in the idle state and did not show any visible increase in memory use during the snapshot phase. At peak, Redis memory use increased to almost 3X that of Dragonfly.

Dragonfly finished the snapshot faster, within a few seconds.

For more info about memory efficiency in Dragonfly, see our Dashtable doc.

<a name="configuration"><a/>Configuration

Dragonfly supports common Redis arguments where applicable. For example, you can run: dragonfly --requirepass=foo --bind localhost.

Dragonfly currently supports the following Redis-specific arguments:

port: Redis connection port (default: 6379).
bind: Use localhost to only allow localhost connections or a public IP address to allow connections to that IP address (i.e. from outside too). Use 0.0.0.0 to allow all IPv4.
requirepass: The password for AUTH authentication (default: "").
maxmemory: Limit on maximum memory (in human-readable bytes) used by the database (default: 0). A maxmemory value of 0 means the program will automatically determine its maximum memory usage.
dir: Dragonfly Docker uses the /data folder for snapshotting by default, the CLI uses "". You can use the -v Docker option to map it to your host folder.
dbfilename: The filename to save and load the database (default: dump).

There are also some Dragonfly-specific arguments:

memcached_port: The port to enable Memcached-compatible API on (default: disabled).
keys_output_limit: Maximum number of returned keys in keys command (default: 8192). Note that keys is a dangerous command. We truncate its result to avoid a blowup in memory use when fetching too many keys.
dbnum: Maximum number of supported databases for select.
cache_mode: See the novel cache design section below.
hz: Key expiry evaluation frequency (default: 100). Lower frequency uses less CPU when idle at the expense of a slower eviction rate.
snapshot_cron: Cron schedule expression for automatic backup snapshots using standard cron syntax with the granularity of minutes (default: ""). Here are some cron schedule expression examples below, and feel free to read more about this argument in our documentation.

| Cron Schedule Expression | Description | |--------------------------|--------------------------------------------| | * * * * * | At every minute | | */5 * * * * | At every 5th minute | | 5 */2 * * * | At minute 5 past every 2nd hour | | 0 0 * * * | At 00:00 (midnight) every day | | 0 6 * * 1-5 | At 06:00 (dawn) from Monday through Friday |
primary_port_http_enabled: Allows accessing HTTP console on main TCP port if true (default: true).
admin_port: To enable admin access to the console on the assigned port (default: disabled). Supports both HTTP and RESP protocols.
admin_bind: To bind the admin console TCP connection to a given address (default: any). Supports both HTTP and RESP protocols.
admin_nopass: To enable open admin access to console on the assigned port, without auth token needed (`default: fal

Dragonfly

Install / Use

README