Distributed Systems in Rust

This project is followed with Talent Plan from PingCAP University. Origin README can find here

Project Workspaces

• labcodec: encode and decode message in the RPC network
• labrpc: simulate a distributed network with multi-node and RPC service between nodes
• linearizability: some test for linearizability (maybe not used)
• percolator:
  • a system built by Google for incremental processing on a very large data set.
  • It also provides a distributed transaction protocol with ACID snapshot-isolation semantics
  • origin paper: Large-scale Incremental Processing Using Distributed Transactions and Notifications
  • see more in percolator/README.md
• raft:
  • a series of labs on a key/value storage system built with the Raft consensus algorithm.
  • These labs are derived from the lab2:raft and lab3:kvraft from the famous MIT 6.824 course but rewritten in Rust.
  • origin paper: In Search of an Understandable Consensus Algorithm (Extended Version)
  • see more in raft/README.md
• report: some log output file

Feature

• similar with MIT 6.824 but rewrite all test code in Rust by Talent Plan
• writen with pure Rust and ProtoBuf protocol
• use prost to parse ProtoBuf message
• use log and env_logger as log format
• largely use futures and tokio to achieve asynchrony

Progress:

• Raft (lab 2)
  • [X] 2A: leader election and heartbeat
  • [X] 2B: log entry and append entries
  • [X] 2C: persist and unliable network
  • [X] All tests passed
• KvRaft (lab 3)
  • [X] 3A: server/client query
  • [X] 3B: log compaction and install snapshot
  • [X] All tests passed
• Percolator
  • [X] Server: TimestampOracle
  • [X] Server: KvTable and MemoryStorage
  • [X] Server: Get, Prewrite, Commit
  • [X] Client: Set, Get, Commit
  • [X] All tests passed

TODO

• Raft and KvRaft
  • [ ] add more comment
  • [ ] prune some condition to make program clear
• Percolator
  • [ ] improve 2PC logic
  • [ ] improve lock conflict and back-off/roll-forward