Unisondb

UnisonDB

Store, stream, and sync instantly — UnisonDB is a log-native, real-time database that replicates like a message bus for AI and Edge Computing.

<img src="docs/logo.svg" width="300" alt="UnisonDB" />

What UnisonDB Is

UnisonDB is an open-source database designed specifically for Edge AI and Edge Computing.

It is a reactive, log-native and multi-model database built for real-time and edge-scale applications. UnisonDB combines a B+Tree storage engine with WAL-based (Write-Ahead Logging) replication over gRPC or S3-compatible blob/object storage, enabling near-instant fan-out replication across hundreds of nodes while preserving strong consistency and durability.

Replication Model

Writes are committed by a Raft quorum on the write servers (if enabled); read-only edge replicas and relayers can consume WAL through either a live gRPC stream or blob-backed replication using S3-compatible object storage.

Blob-backed replication changes the fan-out model:

• The writer publishes WAL durably into object storage
• Any number of readers can poll and catch up directly from the blob store
• Teams already running S3 or MinIO do not need to maintain an always-on gRPC replication path for every replica

See cmd/examples/blobstore-minio for a local MinIO example.

Key Features

• High Availability Writes: Raft consensus on write servers (quorum acks); relayer/replica use in-sync replica (ISR) replication
• Streaming Replication: WAL replication over gRPC or S3-compatible blob/object storage
• Blob Fan-Out: Publish WAL once into object storage and let N readers poll directly from S3/MinIO-backed replication stores
• Multi-Modal Storage: Key-Value, Wide-Column, and Large Objects (LOB)
• Real-Time Notifications: ZeroMQ-based(Side-car) change notifications with sub-millisecond latency
• Durable: B+Tree storage with Write-Ahead Logging
• Edge-First Design: Optimized for edge computing and local-first architectures
• Namespace Isolation: Multi-tenancy support with namespace-based isolation

Use Cases

UnisonDB is built for distributed edge-first architectures systems where data and computation must live close together — reducing network hops, minimizing latency, and enabling real-time responsiveness at scale.

By co-locating data with the services that use it, UnisonDB removes the traditional boundary between the database and the application layer. Applications can react to local changes instantly, while UnisonDB’s WAL-based replication ensures eventual consistency across all replicas globally.

Fan-Out Scaling

UnisonDB can fan out updates to 100+ edge nodes in just a few milliseconds from a single upstream—and because it supports multi-hop relaying, that reach compounds naturally. Each hop carries the network + application latency of its link;

In a simple 2-hop topology:

• Hop 1: Primary → 100 hubs (≈250–500ms)
• Hop 2: Each hub → 100 downstream edge nodes (similar latency)
• Total reach: 100 + 10,000 = 10,100 nodes

Even at 60k–80k SET ops/sec with 1 KB values, UnisonDB can propagate those updates across 10,000+ nodes within seconds—without Kafka, Pub/Sub, CDC pipelines, or heavyweight brokers. (See the Relayer vs Latency benchmarks below for measured numbers.)

Quick Start

# Clone the repository
git clone https://github.com/ankur-anand/unisondb
cd unisondb

# Build
go build -o unisondb ./cmd/unisondb

# Run in server mode (primary)
./unisondb server --config config.toml

# Use the HTTP API
curl -X PUT http://localhost:4000/api/v1/default/kv/mykey \
  -H "Content-Type: application/json" \
  -d '{"value":"bXl2YWx1ZQ=="}'

Documentation

1. Getting Started with UnisonDB
2. Complete Configuration Guide
3. Architecture Overview
4. HTTP API Reference
5. Backup and Restore
6. Deployment Topologies
7. Rough Roadmap

UnisonDB implements a pluggable storage backend architecture supporting two BTree implementations:

• BoltDB: Single-file, ACID-compliant BTree.
• LMDB: Memory-mapped ACID-compliant BTree with copy-on-write semantics.

Redis-Compatible Benchmark: UnisonDB vs BadgerDB vs BoltDB vs LMDB

This benchmark compares the write and read performance of four databases — UnisonDB, BadgerDB, LMDB* and BoltDB — using a Redis-compatible interface and the official redis-benchmark tool.

What We Measured

• Throughput: Requests per second for SET (write) and GET (read) operations
• Latency: p50 latency in milliseconds
• Workload: 50 iterations of mixed SET and GET operations (200k ops per run)
• Concurrency: 10 parallel clients, 10 pipelined requests, 4 threads
• Payload Size: 1KB

Test Environment

Chip: Apple M2 Pro
Total Number of Cores: 10 (6 performance and 4 efficiency)
Memory: 16 GB
Unisondb Btree Backend - LMDB

All three databases were tested under identical conditions to highlight differences in write path efficiency, read performance, and I/O characteristics. The Redis-compatible server implementation can be found in internal/benchtests/cmd/redis-server/.

Results

<img src="docs/ubbl_compare.png" alt="UnisonDB, BadgerDB, BoltDB, LMDB Comparison" />

Performance Testing: Local Replication

Test Setup

We validated the WAL-based replication architecture using the pkg/replicator component. It Uses the same redis-compatible bench tool but this time the server is started a n=[100,200,500,750,1000] goroutine that is an independent WAL reader, capturing critical performance metrics:

1. Physical Latency Tracking: Measures p50, p90, p99, and max latencies Vs Relayer.
2. SET, GET Latency vs Relayer
3. SET, GET Throughput Vs Relayer.

Results

<img src="docs/relayer_set_get.png" alt="relayer_set_get" /> <img src="docs/relayer_latency_granular.png" alt="relayer_latency_granular" />

Test Replication Flow

<img src="./docs/replication_test.png">

---

Why UnisonDB

Traditional databases persist. Stream systems propagate. UnisonDB does both — turning every write into a durable, queryable stream that replicates seamlessly across the edge.

The Problem: Storage and Streaming Live in Different Worlds

Modern systems are reactive — every change needs to propagate instantly to dashboards, APIs, caches, and edge devices.
Yet, databases were built for persistence, not propagation.

You write to a database, then stream through Kafka.
You replicate via CDC.
You patch syncs between cache and storage.

This split between state and stream creates friction:

• Two systems to maintain and monitor
• Eventual consistency between write path and read path
• Network latency on every read or update
• Complex fan-out when scaling to hundreds of edges

---

The Gap

LMDB and BoltDB excel at local speed — but stop at one node.
etcd and Consul replicate state — but are consensus-bound and small-cluster only.
Kafka and NATS stream messages — but aren’t queryable databases.

| System | Strength | Limitation | |---------|-----------|-------------| | LMDB / BoltDB | Fast local storage | No replication | | etcd / Consul | Cluster consistency | No local queries, low fan-out | | Kafka / NATS | Scalable streams | No storage or query model |

---

The Solution: Log-Native by Design

UnisonDB fuses database semantics with streaming mechanics — the log is the database.
Every write is durable, ordered, and instantly available as a replication stream.

No CDC, no brokers, no external pipelines.
Just one unified engine that:

• Stores data in B+Trees for predictable reads
• Streams data via WAL replication to thousands of nodes
• Reacts instantly with sub-second fan-out
• Keeps local replicas fully queryable, even offline

UnisonDB eliminates the divide between “database” and “message bus,”
enabling reactive, distributed, and local-first systems — without the operational sprawl.

UnisonDB collapses two worlds — storage and streaming — into one unified log-native core.
The result: a single system that stores, replicates, and reacts — instantly.

---

Core Architecture

UnisonDB is built on three foundational layers:

1. WALFS - Write-Ahead Log File System (mmap-based, optimized for reading at scale).
2. Engine - Hybrid storage combining WAL, MemTable, and B-Tree
3. **R