Anvil

Anvil is distributed object storage for application state. It keeps opaque bytes at stable paths and supplies the coordination primitives applications otherwise have to assemble around a blob store: streaming writes, compare-and-swap, immutable namespaces, Zanzibar authorization, bounded atomic programs, change notification, and materialized search indexes.

Run one process while developing. Add nodes when you need capacity or availability; clients keep using the same API, any active node can accept a request, and Anvil places data across heterogeneous nodes with capacity-weighted rendezvous hashing.

What is available

| Capability | Release | Status | | --- | --- | --- | | Object storage | 0.5.0 | Streaming puts, deduplication, CAS, immutable puts, bulk writes, batch reads, deletes, optional version retention, prefix listing, and watches | | Authorization | 0.5.0 | Application credentials, short-lived JWTs, protected administration, Zanzibar schemas, tuples, roles, and checks | | Atomic programs | 0.5.0 | Explicitly selected, deterministic multi-path state transitions without routing ordinary uploads through a transaction system | | Distributed clusters | 0.5.1 | Any-node ingress, peer mTLS, replicated metadata, weighted placement, and 2+1 erasure-coded payload durability | | Materialized indexes | 0.5.2 | Path, object metadata, typed JSON, full text, vector, hybrid, Git-source, and tensor indexes | | Rust client | 0.5.2 | Credential exchange, authenticated clients, streaming upload helpers, and the complete generated gRPC API | | PersonalDB, public reads, accounting, S3 and Git | 0.5.3 | Protocol-native PersonalDB groups and projections, authorized usage aggregates, opt-in anonymous reads, and standard S3/Git gateways | | Online cluster growth | 0.5.4 | Large objects use complete replicas below the configured erasure width, then move online to the fixed erasure profile as nodes join | | Shared public listener | 0.5.5 | Native gRPC, S3, Git, and administrative APIs share one authorized public endpoint; peer mTLS remains isolated | | Java client | — | TODO | | Python client | — | TODO | | Node.js client | — | TODO | | Ruby client | — | TODO | | Network plugins | — | Planned after 0.5.3 |

The published container is a single multi-platform image for Linux AMD64 and ARM64.

Your first object in five minutes

This walkthrough starts one node, creates the system administrator, provisions a tenant and its first application, creates a bucket, then writes and reads an object. It uses the CLI inside the published image, so only Docker and this repository are required.

1. Start a development node

export ANVIL_IMAGE=ghcr.io/worka-ai/anvil:0.5.5
export ANVIL_TOKEN_SIGNING_KEY_FILE="$PWD/anvil-data/token-signing-key"

mkdir -p anvil-data
head -c 64 /dev/urandom > "$ANVIL_TOKEN_SIGNING_KEY_FILE"
chmod 0600 "$ANVIL_TOKEN_SIGNING_KEY_FILE"

# The container runs as UID 10001 and deliberately rejects a broadly readable
# signing key.
docker run --rm --user 0 \
  -v "$ANVIL_TOKEN_SIGNING_KEY_FILE:/key" \
  "$ANVIL_IMAGE" chown 10001:10001 /key

ANVIL_RUN_SYSTEM_BOOTSTRAP=true \
  docker compose -f crates/anvil/docker-compose.yml up -d

The first successful bootstrap creates one mode-0600 credential at /var/lib/anvil/system-bootstrap-credential.json. It belongs to the protected system administrator application. Copy it to an operator secret store, then remove the generated copy:

docker compose -f crates/anvil/docker-compose.yml cp \
  anvil:/var/lib/anvil/system-bootstrap-credential.json \
  anvil-data/system-bootstrap-credential.json
chmod 0600 anvil-data/system-bootstrap-credential.json

Bootstrap is explicit and one-shot. Recreate the process without the flag after the first successful start; the named data volume is retained:

ANVIL_RUN_SYSTEM_BOOTSTRAP=false \
  docker compose -f crates/anvil/docker-compose.yml up -d --force-recreate

2. Create a tenant and its owner application

An application authenticates with a client_id and client_secret. Credential exchange returns a short-lived bearer token; the CLI and Rust client perform that exchange for you.

Choose and retain a strong application secret:

export ANVIL_OWNER_SECRET="$(openssl rand -hex 32)"

Use the system credential to create tenant example, owner application example-owner, and client example-client in one operation:

docker compose -f crates/anvil/docker-compose.yml exec \
  -e ANVIL_NEW_CLIENT_SECRET="$ANVIL_OWNER_SECRET" anvil \
  anvil --endpoint http://127.0.0.1:50051 \
  --credentials-file /var/lib/anvil/system-bootstrap-credential.json \
  provision-tenant example example-owner example-client

The generated system credential should not remain on the server after you have copied it and completed bootstrap:

docker compose -f crates/anvil/docker-compose.yml exec --user 0 anvil \
  rm /var/lib/anvil/system-bootstrap-credential.json

3. Create a bucket

The tenant owner may create buckets. The application that creates a bucket becomes its owner:

docker compose -f crates/anvil/docker-compose.yml exec \
  -e ANVIL_CLIENT_ID=example-client \
  -e ANVIL_CLIENT_SECRET="$ANVIL_OWNER_SECRET" anvil \
  anvil --endpoint http://127.0.0.1:50051 \
  create-bucket objects

Buckets are unversioned by default: overwriting or deleting a current object does not expose an older value later. Use create-bucket objects --versioning enabled when retained versions are part of the application contract.

4. Upload and read an object

printf 'hello from Anvil\n' > anvil-data/hello.txt
docker compose -f crates/anvil/docker-compose.yml cp \
  anvil-data/hello.txt anvil:/tmp/hello.txt

docker compose -f crates/anvil/docker-compose.yml exec \
  -e ANVIL_CLIENT_ID=example-client \
  -e ANVIL_CLIENT_SECRET="$ANVIL_OWNER_SECRET" anvil \
  anvil --endpoint http://127.0.0.1:50051 \
  put example objects greetings/hello.txt /tmp/hello.txt \
  --content-type text/plain --command-id first-upload

docker compose -f crates/anvil/docker-compose.yml exec \
  -e ANVIL_CLIENT_ID=example-client \
  -e ANVIL_CLIENT_SECRET="$ANVIL_OWNER_SECRET" anvil \
  anvil --endpoint http://127.0.0.1:50051 \
  get example objects greetings/hello.txt

The last command prints hello from Anvil. You now have a tenant-isolated, Zanzibar-authorized object addressed by (tenant, bucket, path).

Use the Rust client

cargo add anvil-storage@0.5.5
cargo add tokio --features macros,rt-multi-thread

The application needs the public endpoint and the client ID and secret created above. Tenant and bucket are part of each object address, not connection-wide state.

use anvil_storage::v1::{
    Durability, HeadObjectRequest, ObjectAddress, PutHeader, PutOperation,
    put_header,
};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
    let mut objects = anvil_storage::connect_with_credentials(
        "http://127.0.0.1:50051",
        "example-client",
        std::env::var("ANVIL_OWNER_SECRET")?,
    )
    .await?;

    let address = ObjectAddress {
        tenant: "example".into(),
        bucket: "objects".into(),
        path: "greetings/from-rust.txt".into(),
    };

    let receipt = anvil_storage::put_chunks(
        &mut objects,
        PutHeader {
            address: Some(address.clone()),
            content_type: "text/plain".into(),
            command_id: "rust-first-upload".into(),
            durability: Durability::Local as i32,
            operation: Some(put_header::Operation::Put(PutOperation {})),
        },
        [b"hello from Rust\n".to_vec()],
    )
    .await?;

    let head = objects
        .head_object(HeadObjectRequest {
            address: Some(address),
        })
        .await?
        .into_inner();

    println!("published version {}: {head:?}", receipt.version);
    Ok(())
}

LOCAL and REPLICATED describe when the client is acknowledged, not where the object ultimately lives. LOCAL returns after the ingress node has durably accepted the write while normal placement continues. REPLICATED waits for the fixed 2+1 payload guarantee (or the corresponding mutable-record quorum).

For long-running processes, exchange credentials again when the returned token expires. connect_channel, exchange_client_credentials, BearerToken, and the generated service clients let an application share one transport across object, index, authorization, and administration calls. The focused Rust guide is at clients/rust/README.md.

Create a PersonalDB group

PersonalDB gives an application a witnessed, predecessor-linked log for SQLite changesets. Source and standalone groups accept authorized appends; projection groups are materialized explicitly from a source group. Group roles are Zanzibar-authorized independently of ordinary object traffic.

Add the public client and canonical protocol types:

cargo add anvil-storage@0.5.5 personaldb-protocol@0.2.2 serde_json

Use the same application credential created above to create a source group and verify Anvil's signed descriptor:

use anvil_storage::v1::{CreatePersonalDbGroupRequest, PersonalDbGroupKind};
use personaldb_protocol::{
    GroupDescriptor, PublicKeyTrustRecord, PublicKeyTrustStore, Sha256Digest,
};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
    let channel = anvil_storage::connect_channel("http://127.0.0.1:50051").await?;
    let token = anvil_storage::exchange_client_credentials(
        channel.clone(),
        "example-client",
        std::env::var("ANVIL_OWNER_SECRET")?,
    )
    .await?;
    let mut personaldb = anvil_storage::personaldb_client(channel, &token.access_token)?;

    let group = personaldb
        .create_group(CreatePersonalDbGroupRequest {
            bucket: "objects".into(),
            database_id: "main".into(),