GitHub Release

Wildcat is a high-performance embedded key-value database (or storage engine) written in Go with C interoperability. It incorporates modern database design principles including LSM (Log-Structured Merge) tree architecture, MVCC (Multi-Version Concurrency Control), and lock-free data structures for its critical paths, along with automatic background operations to deliver excellent read/write performance with immediate consistency and durability.

Features

LSM (Log-Structured Merge) tree architecture optimized for write-heavy workloads
Mostly lock-free MVCC with minimal blocking on critical paths
WAL logging captures full transaction state for recovery and rehydration
Version-aware skip list for fast in-memory MVCC access
Thread-safe write operations with atomic coordination
Scalable design with background flusher and compactor
Concurrent block storage leveraging direct, offset-based file I/O (using pread/pwrite) for optimal performance
Atomic LRU cache for active block manager handles
Atomic memtable lifecycle management
SSTables are immutable BTrees
Configurable durability levels None (fastest), Partial (balanced), Full (most durable)
Snapshot-isolated MVCC with timestamp-based reads
Crash recovery preserves committed transactions and maintains access to incomplete transactions if configured with RecoverUncommittedTxns
Automatic multi-threaded background compaction with configurable concurrency
ACID transaction support with configurable durability guarantees
Range, prefix, and full iteration support with bidirectional traversal
High transactional throughput per second with low latency due to lock-free and non-blocking design from memory to disk
Optional Bloom filters per SSTable for improved key lookup performance
Key-value separation optimization (.klog for keys, .vlog for values)
Tombstone-aware and version-aware compaction with retention based on active transaction read windows
Transaction recovery preserves incomplete transactions for post-crash inspection and resolution if db configured with RecoverUncommittedTxns
Keys and values stored as opaque byte sequences
Single-node embedded storage engine with no network or replication overhead

Discord Community

Join our Discord community to discuss development, design, ask questions, and get help with Wildcat.

Version and Compatibility
Basic Usage
Shared C Library
Contributing

Samples

You can find sample programs here

Benchmarks

You can find the benchmarking tool here

Wiki

You can read about Wildcat's architecture at the wiki.

Version and Compatibility

Go 1.24+
Linux/macOS/Windows (64-bit)

Basic Usage

Wildcat supports opening multiple *wildcat.DB instances in parallel, each operating independently in separate directories.

Downloading

go get github.com/wildcatdb/wildcat/v2

Specific major version i.e v2.x.x can be downloaded using

go get github.com/wildcatdb/wildcat/v2@v2.x.x

Import

import (
    "github.com/wildcatdb/wildcat/v2"
)

When importing different majors you can do

// v1 (not recommended, use v2+)
import (
    "github.com/wildcatdb/wildcat"
)

// v2
import (
    "github.com/wildcatdb/wildcat/v2"
)

// v2+
import (
    "github.com/wildcatdb/wildcat/v32"
)

Opening a Wildcat DB instance

The only required option is the database directory path.

// Create default options
opts := &wildcat.Options{
    Directory: "/path/to/db",
    // You don't need to set all options only Directory is required!
}

// Open or create a new Wildcat DB instance
db, err := wildcat.Open(opts) // Returns *wildcat.DB
if err != nil {
    // Handle error
}
defer db.Close()

Simple Key-Value Operations

The easiest way to interact with Wildcat is through the Update method, which handles transactions automatically. This means it runs begin, commit, and rollback for you, allowing you to focus on the operations themselves.

// Write a value
err := db.Update(func(txn *wildcat.Txn) error {
    return txn.Put([]byte("hello"), []byte("world")) // Put update's existing key's values
})
if err != nil {
    // Handle error
}

// Read a value
var result []byte
err = db.View(func(txn *wildcat.Txn) error {
    var err error
    result, err = txn.Get([]byte("hello"))
    return err
})

if err != nil {
    // Handle error
} else {
    fmt.Println("Value:", string(result)) // Outputs: Value: world
}

Manual Transaction Management

For more complex operations, you can manually manage transactions.

txn, err := db.Begin()
if err != nil {
    // Handle error
    return
}

// Perform operations
err := txn.Put([]byte("key1"), []byte("value1"))
if err != nil {
    txn.Rollback()
    // Handle error
}

value, err := txn.Get([]byte("key1"))
if err != nil {
    txn.Rollback()
    // Handle error
}

// Commit or rollback
err = txn.Commit()
if err != nil {
    txn.Rollback()
    // Handle commit error
}

Read-Only Transactions with View

// Read a value with View
var result []byte
err = db.View(func(txn *wildcat.Txn) error {
    var err error
    result, err = txn.Get([]byte("hello"))
    return err
})

Batch Operations

You can perform multiple operations in a single transaction.

[!CAUTION] Batch operations on the Wildcat engine are slower completed inside an Update. It's better to use Begin->Put flow for batch writes.

err := db.Update(func(txn *wildcat.Txn) error {
    // Write multiple key-value pairs
    for i := 0; i < 1000; i++ {
        key := []byte(fmt.Sprintf("key%d", i))
        value := []byte(fmt.Sprintf("value%d", i))

        if err := txn.Put(key, value); err != nil {
            return err
        }
    }
    return nil
})

// Perform batch operations
for i := 0; i < 1000; i++ {
    txn, err := db.Begin()
    if err != nil {
        // Handle error
        return
    }

    key := []byte(fmt.Sprintf("key%d", i))
    value := []byte(fmt.Sprintf("value%d", i))

    if err := txn.Put(key, value); err != nil {
        txn.Rollback()
        // Handle error
        return
    }

    err = txn.Commit()
    if err != nil {
        // Handle error
        return
    }
}

Iterating Keys

Wildcat provides comprehensive iteration capabilities with MVCC consistency.

[!TIP] You can set ascending or descending order, and iterate over all keys, a range of keys, or keys with a specific prefix.

Full Iterator (bidirectional)

err := db.View(func(txn *wildcat.Txn) error {
    // Create ascending iterator
    iter, err := txn.NewIterator(true)
    if err != nil {
        return err
    }

    // Iterate forward
    for {
        key, value, timestamp, ok := iter.Next()
        if !ok {
            break
        }

        fmt.Printf("Key: %s, Value: %s, Timestamp: %d\n", key, value, timestamp)
    }

    // Change direction and iterate backward
    err = iter.SetDirection(false)
    if err != nil {
        return err
    }

    for {
        key, value, timestamp, ok := iter.Next()
        if !ok {
            break
        }

        fmt.Printf("Key: %s, Value: %s, Timestamp: %d\n", key, value, timestamp)
    }

    return nil
})

Range Iterator (bidirectional)

err := db.View(func(txn *wildcat.Txn) error {
    iter, err := txn.NewRangeIterator([]byte("start"), []byte("end"), true)
    if err != nil {
        return err
    }

    // Iterate forward
    for {
        key, value, timestamp, ok := iter.Next()
        if !ok {
            break
        }

        fmt.Printf("Key: %s, Value: %s, Timestamp: %d\n", key, value, timestamp)
    }

    // Change direction and iterate backward
    err = iter.SetDirection(false)
    if err != nil {
        return err
    }

    for {
        key, value, timestamp, ok := iter.Next()
        if !ok {
            break
        }

        fmt.Printf("Key: %s, Value: %s, Timestamp: %d\n", key, value, timestamp)
    }

    return nil
})

Prefix Iterator (bidirectional)

err := db.View(func(txn *wildcat.Txn) error {
    iter, err := txn.NewPrefixIterator([]byte("prefix"), true)
    if err != nil {
        return err
    }

    // Iterate forward
    for {
        key, value, timestamp, ok := iter.Next()
        if !ok {
            break
        }

        fmt.Printf("Key: %s, Value: %s, Timestamp: %d\n", key, value, timestamp)
    }

    // Change direction and iterate backward
    err = iter.SetDirection(false)
    if err != nil {
        return err
    }

    for {
        key, val

Wildcat

Install / Use

README