OxideDB

⚠️ Disclaimer This project was developed entirely through AI-assisted programming using Anthropic's Claude (claude-4.6-opus) via Cursor IDE. OxideDB is an experimental, proof-of-concept database engine created for educational and research purposes. It is not intended for production use. No guarantees are made regarding data integrity, performance, security, or availability in real-world environments. Use at your own risk.

A Couchbase-compatible NoSQL database written entirely in Rust. Features full Couchbase SDK compatibility (Python, Go), XDCR replication, B+ Tree persistent storage with WAL, secondary indexes, N1QL-like query language, multi-node cluster support with auto-failover, and a built-in web management console.

Features

| Feature | Description | |---------|-------------| | Couchbase SDK Compatible | Works with official Couchbase Python SDK and Go SDK via Memcached binary protocol | | XDCR Replication | Cross-Datacenter Replication with conflict resolution (Sequence Number & LWW/Timestamp) | | B+ Tree Storage | 4KB page-based B+ Tree persistent storage engine with binary encoding | | Write-Ahead Log | Buffered WAL with dual-trigger flush (ops count, byte size, time interval) | | 1024 vBuckets | CRC32-based consistent hashing across 1024 virtual buckets | | Secondary Indexes | GSI-like Global Secondary Indexes with composite key and nested field support | | N1QL Query Language | SQL-like query language with SELECT, WHERE, ORDER BY, LIMIT, CREATE INDEX | | Multi-Node Cluster | Node management, heartbeat monitoring, partition rebalancing | | Auto-Failover | Automatic failure detection with configurable timeout, quotas, and recovery | | Bucket/Scope/Collection | Full Couchbase data hierarchy: Buckets → Scopes → Collections → Documents | | Memcached Binary Protocol | Full KV protocol: GET, SET, ADD, REPLACE, DELETE, INCREMENT, DECREMENT, APPEND, PREPEND | | SASL Authentication | PLAIN, SCRAM-SHA512, SCRAM-SHA256 authentication mechanisms | | REST API | Comprehensive REST API for all management and data operations | | Full-Text Search (FTS) | Inverted index with BM25 scoring, match/phrase/term/prefix/wildcard/bool queries, highlighting | | DCP (Database Change Protocol) | Real-time mutation streaming with SSE, backfill, multi-stream, broadcast channel | | Web UI Console | Built-in web management console with dashboard, document browser, query workbench, FTS, DCP | | TTL Support | Document-level and bucket-level expiry with automatic purge | | CAS (Compare-and-Swap) | Optimistic concurrency control on all mutations |

Quick Start

Prerequisites

Rust 1.76.0 or later
Cargo (comes with Rust)

Build & Run

# Clone the repository
git clone https://github.com/your-org/oxidedb.git
cd oxidedb

# Build in release mode
cargo build --release

# Run with defaults (REST: 8091, KV: 11210)
./target/release/oxidedb

# Or with custom configuration
./target/release/oxidedb \
  --port 8091 \
  --memcached-port 11210 \
  --data-dir ./data \
  --node-name node-1 \
  --num-vbuckets 1024

Using Docker (Optional)

FROM rust:1.76 AS builder
WORKDIR /app
COPY . .
RUN cargo build --release

FROM debian:bookworm-slim
COPY --from=builder /app/target/release/oxidedb /usr/local/bin/
EXPOSE 8091 11210
CMD ["oxidedb"]

Architecture

┌─────────────────────────────────────────────────────────┐
│                     Client Layer                         │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐  │
│  │ Couchbase SDK │  │   REST API   │  │    Web UI    │  │
│  │ (Python/Go)  │  │  (Axum)      │  │  (Embedded)  │  │
│  └──────┬───────┘  └──────┬───────┘  └──────┬───────┘  │
└─────────┼──────────────────┼─────────────────┼──────────┘
          │                  │                 │
┌─────────┼──────────────────┼─────────────────┼──────────┐
│         ▼                  ▼                 ▼          │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐  │
│  │  Memcached   │  │   API Layer  │  │  Query       │  │
│  │  Binary      │  │  (Routes &   │  │  Engine      │  │
│  │  Protocol    │  │   Handlers)  │  │  (N1QL-like) │  │
│  │  (port 11210)│  │  (port 8091) │  │              │  │
│  └──────┬───────┘  └──────┬───────┘  └──────┬───────┘  │
│         │                  │                 │          │
│         ▼                  ▼                 ▼          │
│  ┌──────────────────────────────────────────────────┐   │
│  │              Storage Engine                       │   │
│  │  ┌────────────────────────────────────────────┐  │   │
│  │  │  Bucket → Scope → Collection → Document    │  │   │
│  │  └────────────────────────────────────────────┘  │   │
│  │  ┌─────────────┐  ┌──────────────────────────┐  │   │
│  │  │  vBuckets    │  │  Secondary Indexes (GSI) │  │   │
│  │  │  (1024 ×     │  │  BTreeMap-based          │  │   │
│  │  │   CRC32)     │  │  Composite keys          │  │   │
│  │  └──────┬──────┘  └──────────────────────────┘  │   │
│  │         │                                        │   │
│  │  ┌──────▼──────────────────────────────────┐    │   │
│  │  │  Persistence Layer                       │    │   │
│  │  │  ┌───────────┐  ┌───────────────────┐   │    │   │
│  │  │  │  B+ Tree   │  │  WAL (Write-Ahead │   │    │   │
│  │  │  │  (4KB pages,│  │  Log) + Buffered  │   │    │   │
│  │  │  │   binary)  │  │  Dual-trigger      │   │    │   │
│  │  │  └───────────┘  └───────────────────┘   │    │   │
│  │  └─────────────────────────────────────────┘    │   │
│  └──────────────────────────────────────────────────┘   │
│                                                          │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐  │
│  │   Cluster    │  │    XDCR      │  │  Auto-       │  │
│  │   Manager    │  │  Replicator  │  │  Failover    │  │
│  └──────────────┘  └──────────────┘  └──────────────┘  │
└─────────────────────────────────────────────────────────┘

Data Model

Cluster
 └── Bucket (e.g. "travel-sample")
      ├── BucketConfig (type, RAM quota, replicas, conflict resolution)
      ├── Scopes
      │    ├── _default (auto-created)
      │    │    └── Collections
      │    │         └── _default (auto-created)
      │    └── inventory
      │         ├── airlines
      │         ├── airports
      │         └── routes
      └── vBuckets [0..1023]
           └── Documents
                ├── key: "airline_123"
                ├── value: { JSON }
                ├── CAS: 1707123456789
                ├── seq_no: 42
                ├── rev_id: 3
                ├── expiry: Optional<DateTime>
                ├── flags: 0
                └── vbucket_id: 587

Configuration

All configuration is via CLI arguments:

| Argument | Default | Description | |----------|---------|-------------| | --host | 0.0.0.0 | Bind address | | --port / -p | 8091 | REST API port | | --memcached-port | 11210 | Memcached binary protocol port (SDK) | | --data-dir | ./data | Directory for persistent storage | | --node-name | node-1 | Unique name for this cluster node | | --num-vbuckets | 1024 | Number of virtual buckets per bucket | | --enable-persistence | true | Enable disk persistence | | --ttl-check-interval-secs | 1 | How often to check for expired documents | | --xdcr-replication-interval-ms | 500 | XDCR replication cycle interval | | --wal-buffer-max-ops | 5000 | Flush WAL after N operations | | --wal-buffer-max-bytes | 4194304 | Flush WAL after N bytes (4MB) | | --wal-flush-interval-ms | 1000 | Flush WAL every N milliseconds | | --btree-compact-interval-secs | 30 | B+ Tree compaction interval |

Environment Variables

| Variable | Description | |----------|-------------| | RUST_LOG | Log level: error, warn, info, debug, trace |

Example: Production Setup

RUST_LOG=info ./oxidedb \
  --port 8091 \
  --memcached-port 11210 \
  --data-dir /var/lib/oxidedb \
  --node-name prod-node-1 \
  --num-vbuckets 1024 \
  --wal-buffer-max-bytes 16777216 \
  --wal-buffer-max-ops 10000 \
  --btree-compact-interval-secs 60

Couchbase SDK Compatibility

Python SDK

from couchbase.cluster import Cluster
from couchbase.options import ClusterOptions
from couchbase.auth import PasswordAuthenticator
from datetime import timedelta

# Connect
auth = PasswordAuthenticator("Administrator", "password")
cluster = Cluster("couchbase://localhost", ClusterOptions(auth))
cluster.wait_until_ready(timedelta(seconds=10))

# Open bucket
bucket = cluster.bucket("my-bucket")
collection = bucket.default_collection()

# CRUD operations
collection.upsert("doc-1", {"name": "Alice", "age": 30})
result = collection.get("doc-1")
print(result.content_as[dict])
collection.remove("doc-1")

Go SDK

package main

import (
    "fmt"
    "time"
    "github.com/couchbase/gocb/v2"
)

func main() {
    cluster, _ := gocb.Connect("couchbase://localhost", gocb.ClusterOptions{
        Authenticator: gocb.PasswordAuthenticator{
            Username: "Administrator",
            Password: "password",
        },
    })
    cluster.WaitUntilReady(10*time.Second, nil)

    bucket := cluster.Bucket("my-bucket")
    bucket.WaitUntilReady(5*time.Second, nil)
    collection := bucket.DefaultCollection()

    // Upsert
    collection.Upsert("doc-1", map[string]interface{}{
        "name": "Bob",
        "age":  25,
    }, nil)

    // Get
    result, _ := collection.Get("doc-1", nil)
    var content map[string]interface{}
    result.Content(&content)
    fmt.Println(content)
}

Supported SDK Operations

| Operation | Status | |-----------|--------| | GET | ✅ | | SET (Upsert) | ✅ | | ADD (Insert) | ✅ | | REPLACE | ✅ | | DELETE (Remove) | ✅ | | INCREMENT | ✅ | | DECREMENT | ✅ | | APPEND | ✅ | | PREPEND | ✅ | | TOUCH | ✅ | | GAT (Get a

Oxidedb

Install / Use

README