Membrane

A general-purpose selective learning and memory substrate for LLM and agentic systems.

Membrane gives long-lived LLM agents structured, revisable memory with built-in decay, trust-gated retrieval, and audit trails. Instead of an append-only context window or flat text log, agents get typed memory records that can be consolidated, revised, contested, and pruned over time.

---

Table of Contents

• Why Membrane
• 60-Second Mental Model
• Key Features
• Memory Types
• Quick Start
• Architecture
• Configuration
• gRPC API
• Revision Operations
• Evaluation and Metrics
• Observability
• TypeScript Client
• LLM Integration Pattern
• Python Client
• Documentation
• Contributing
• Star History
• License

---

Why Membrane

Most LLM/agent "memory" is either ephemeral (context windows that reset each turn) or an append-only text log stuffed into a RAG pipeline. That gives you retrieval, but not learning: facts get stale, procedures drift, and the system cannot revise itself safely.

Membrane makes memory selective and revisable. It captures raw experience, promotes it into structured knowledge, and lets you supersede, fork, contest, or retract that knowledge with evidence. The result is an agent that can improve over time while remaining predictable, auditable, and safe.

60-Second Mental Model

1. Ingest events, tool outputs, observations, and working state.
2. Consolidate episodic traces into semantic facts, competence records, and plan graphs.
3. Retrieve in layers with trust gating and salience ranking.
4. Revise knowledge with explicit operations and audit trails.
5. Decay salience over time unless reinforced by success.

Key Features

• Typed Memory -- Explicit schemas and lifecycles for each memory type, not a flat text store.
• Revisable Knowledge -- Supersede, fork, retract, merge, and contest records with full provenance tracking.
• Competence Learning -- Agents learn how to solve problems (procedures, success rates), not just what happened.
• Decay and Consolidation -- Time-based salience decay keeps memory useful; background consolidation extracts semantic facts, competence records, and plan graphs from episodic traces.
• LLM-Based Semantic Extraction -- On the Postgres + LLM tier, episodic records can be converted into typed semantic facts asynchronously through a structured extraction pipeline.
• Trust-Aware Retrieval -- Sensitivity levels (public, low, medium, high, hyper) with graduated access control and redacted responses for records above the caller's trust level.
• Security and Operations -- SQLCipher encryption at rest, optional TLS and API key authentication, configurable rate limiting, full audit logs.
• Observability -- Built-in metrics for retrieval usefulness, competence success rate, plan reuse frequency, memory growth, and revision rate.
• gRPC API -- 15-method gRPC service with TypeScript and Python client SDKs, or use Membrane as an embedded Go library.
• Vector-Aware Retrieval -- With the Postgres + pgvector backend enabled, competence and plan-graph applicability can be scored with embedding similarity instead of the confidence-only fallback.
• LLM-Ready Context Retrieval -- Retrieve trust-filtered, typed memory and inject it directly into LLM prompts for planning, execution, self-correction, and background learning loops.

Memory Types

| Type | Purpose | Example | |------|---------|---------| | Episodic | Raw experience capture (immutable) | Tool calls, errors, observations from a debugging session | | Working | Current task state | "Backend initialized, frontend pending, docs TODO" | | Semantic | Stable facts and preferences | "User prefers Go for backend services" | | Competence | Learned procedures with success tracking | "To fix linker cache error: clear cache, rebuild with flags" | | Plan Graph | Reusable solution structures as directed graphs | Multi-step project setup workflow with dependencies and checkpoints |

Each memory type has its own schema, lifecycle rules, and consolidation behavior. Episodic records are immutable once ingested. Working memory tracks in-flight task state. Semantic, competence, and plan graph records are the durable output of consolidation and can be revised through explicit operations.

Quick Start

Prerequisites

• Go 1.22 or later
• Make
• Protocol Buffers compiler (protoc >= 3.20) for gRPC development
• Node.js 20+ for the TypeScript client SDK
• Python 3.10+ for the Python client SDK

Build and Run

git clone https://github.com/GustyCube/membrane.git
cd membrane

# Build the daemon
make build

# Run tests
make test

# Start with default SQLite storage
./bin/membraned

# Start with Postgres + pgvector instead
./bin/membraned --postgres-dsn postgres://membrane:membrane@localhost:5432/membrane_test?sslmode=disable

# With custom configuration
./bin/membraned --config /path/to/config.yaml

# Override database path or listen address
./bin/membraned --db /path/to/membrane.db --addr :8080

Using the Go Library

Membrane can be used as an embedded library without running the daemon:

package main

import (
    "context"
    "fmt"
    "log"

    "github.com/GustyCube/membrane/pkg/ingestion"
    "github.com/GustyCube/membrane/pkg/membrane"
    "github.com/GustyCube/membrane/pkg/retrieval"
    "github.com/GustyCube/membrane/pkg/schema"
)

func main() {
    cfg := membrane.DefaultConfig()
    cfg.DBPath = "my-agent.db"

    m, err := membrane.New(cfg)
    if err != nil {
        log.Fatal(err)
    }
    defer m.Stop()

    ctx := context.Background()
    m.Start(ctx)

    // Ingest an episodic event (tool call observation)
    rec, _ := m.IngestEvent(ctx, ingestion.IngestEventRequest{
        Source:    "build-agent",
        EventKind: "tool_call",
        Ref:       "build#42",
        Summary:   "Executed go build, failed with linker error",
        Tags:      []string{"build", "error"},
    })
    fmt.Printf("Ingested episodic record: %s\n", rec.ID)

    // Ingest a semantic observation
    m.IngestObservation(ctx, ingestion.IngestObservationRequest{
        Source:    "build-agent",
        Subject:   "user",
        Predicate: "prefers_language",
        Object:    "go",
        Tags:      []string{"preferences"},
    })

    // Ingest working memory state
    m.IngestWorkingState(ctx, ingestion.IngestWorkingStateRequest{
        Source:     "build-agent",
        ThreadID:   "session-001",
        State:      schema.TaskStateExecuting,
        NextActions: []string{"run tests", "deploy"},
    })

    // Retrieve with trust context
    resp, _ := m.Retrieve(ctx, &retrieval.RetrieveRequest{
        TaskDescriptor: "fix build error",
        Trust: &retrieval.TrustContext{
            MaxSensitivity: schema.SensitivityMedium,
            Authenticated:  true,
        },
        MemoryTypes: []schema.MemoryType{
            schema.MemoryTypeCompetence,
            schema.MemoryTypeSemantic,
        },
    })

    for _, r := range resp.Records {
        fmt.Printf("Found: %s (type=%s, confidence=%.2f)\n", r.ID, r.Type, r.Confidence)
    }
}

Architecture

Membrane runs as a long-lived daemon or embedded library. The architecture is organized into three logical planes:

+------------------+     +------------------+     +----------------------+
|  Ingestion Plane |---->|   Policy Plane   |---->| Storage & Retrieval  |
+------------------+     +------------------+     +----------------------+
        |                        |                         |
   Events, tool            Classification,            SQLCipher (encrypted),
   outputs, obs.,          sensitivity,               audit trails,
   working state           decay profiles             trust-gated access

Storage Model

• Authoritative Store -- SQLite with SQLCipher remains the default embedded store; Postgres + pgvector is available as an opt-in backend for concurrent deployments and embedding-backed retrieval.
• Structured Payloads -- Type-specific