<div align="center"> <h1>OpenRaft</h1> <h4> Advanced <a href="https://raft.github.io/">Raft</a> in 🦀 Rust using <a href="https://tokio.rs/">Tokio</a>. Please ⭐ on <a href="https://github.com/databendlabs/openraft">GitHub</a>! </h4>

<br/>

</div>

This project intends to improve raft as the next-generation consensus protocol for distributed data storage systems (SQL, NoSQL, KV, Streaming, Graph ... or maybe something more exotic).

Currently, openraft is the consensus engine of meta-service cluster in databend.

• 🚀 Get started:

  • OpenRaft guide is the best place to get started,
  • OpenRaft docs for more in-depth details,
  • OpenRaft FAQ explains some common questions.
  • OpenRaft on DeepWiki provides detailed architectural documentation to help understand OpenRaft internals.

• 💡 Example Applications:

  • Examples with OpenRaft 0.9 require OpenRaft 0.9 on crate.io/openraft
  • Examples with OpenRaft 0.10 require OpenRaft 0.10, which is not yet published on crate.io;

• 🙌 Questions?

  • Why not take a peek at our FAQ? You might find just what you need.
  • Wanna chat? Come hang out with us on Discord!
  • Or start a new discussion over on GitHub.
  • Or join our Feishu group.
  • And hey, if you're on WeChat, add us: drmingdrmer. Let's get the conversation started!

Whatever your style, we're here to support you. 🚀 Let's make something awesome together!

• OpenRaft is derived from async-raft with several bugs fixed: Fixed bugs.

Status

• The features are almost complete for building an application.
• Performance: Supports 70,000 writes/sec for a single writer, and 5,600,000 writes/sec with batch writes. See: Performance
• Unit test coverage stands at 92%.
• The chaos test has not yet been completed, and further testing is needed to ensure the application's robustness and reliability.

API status

• OpenRaft API is not stable yet. Before 1.0.0, an upgrade may contain incompatible changes. Check our change-log. A commit message starts with a keyword to indicate the modification type of the commit:

  • DataChange: on-disk data types changes, which may require manual upgrade.
  • Change: if it introduces incompatible changes.
  • Feature: if it introduces compatible non-breaking new features.
  • Fix: if it just fixes a bug.

Versions

• Branch main has been under active development. The main branch is for the release-0.10.

• Branch release-0.9: Latest: ( v0.9.0 | Change log ); Upgrade guide: ⬆️ 0.8 to 0.9; release-0.9 Won't accept new features but only bug fixes.

• Branch release-0.8: Latest: ( v0.8.8 | Change log ); Upgrade guide: ⬆️ 0.7 to 0.8, ⬆️ 0.8.3 to 0.8.4; release-0.8 Won't accept new features but only bug fixes.

• Branch release-0.7: Latest: ( v0.7.6 | Change log ); Upgrade guide: ⬆️ 0.6 to 0.7; release-0.7 Won't accept new features but only bug fixes.

• Branch release-0.6: Latest: ( v0.6.8 | Change log ); release-0.6 won't accept new features but only bug fixes.

Roadmap

• [x] 2022-10-31 Extended joint membership
• [x] 2023-02-14 Minimize confliction rate when electing; See: OpenRaft Vote design; Or use standard raft mode with feature flag single-term-leader.
• [x] 2023-04-26 Goal performance is 1,000,000 put/sec.
• [ ] Reduce the complexity of vote and pre-vote: get rid of pre-vote RPC;
• [ ] Support flexible quorum, e.g.: Hierarchical Quorums
• [ ] Consider introducing read-quorum and write-quorum, improve efficiency with a cluster with an even number of nodes.

<!-- - - [ ] Consider separating log storage and log order storage. - Leader only determines and replicates the index of log entries, not log - payload. -->

Performance

The benchmark is focused on the OpenRaft framework itself and is run on a minimized store and network. This is NOT a real world application benchmark!!!

Single writes:

| clients | put/s | | --: | --: | | 4096 | 3,548,000 | | 1024 | 3,006,000 | | 256 | 1,808,000 | | 64 | 912,000 | | 1 | 33,000 |

Batch writes (4 entries per batch):

| clients | put/s | | --: | --: | | 4096 | 5,615,000 |

For benchmark detail, go to the ./benchmarks/minimal folder.

Features

• Async and Event-Driven: Operates based on Raft events without reliance on periodic ticks, optimizing message batching for high throughput.
• Extensible Storage and Networking: Customizable via RaftLogStorage, RaftStateMachine and RaftNetwork traits, allowing flexibility in choosing storage and network solutions.
• Unified Raft API: Offers a single Raft type for creating and interacting with Raft tasks, with a straightforward API.
• Cluster Formation: Provides strategies for initial cluster setup as detailed in the cluster formation guide.
• Built-In Tracing Instrumentation: The codebase integrates tracing for logging and distributed tracing, with the option to [set verbosi