Gnata

<p align="center"> <picture> <source media="(prefers-color-scheme: dark)" srcset="assets/gnata-dark.png" /> <source media="(prefers-color-scheme: light)" srcset="assets/gnata-light.png" /> <img src="assets/gnata-light.png" alt="gnata" width="720" /> </picture> </p> <p align="center"> A full JSONata 2.x implementation in Go, built for production streaming workloads. </p> <p align="center"> <a href="https://github.com/RecoLabs/gnata/actions/workflows/ci.yml"><img src="https://github.com/RecoLabs/gnata/actions/workflows/ci.yml/badge.svg" alt="CI" /></a> <a href="https://pkg.go.dev/github.com/recolabs/gnata"><img src="https://pkg.go.dev/badge/github.com/recolabs/gnata.svg" alt="Go Reference" /></a> <a href="https://github.com/RecoLabs/gnata/blob/main/go.mod"><img src="https://img.shields.io/github/go-mod/go-version/RecoLabs/gnata" alt="Go version" /></a> <a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="MIT License" /></a> <a href="https://www.npmjs.com/package/gnata-js"><img src="https://img.shields.io/npm/v/gnata-js" alt="npm version" /></a> </p> <p align="center"> <a href="#quick-start">Quick Start</a> &middot; <a href="#streaming-api">Streaming API</a> &middot; <a href="#metrics--observability">Metrics</a> &middot; <a href="#performance">Performance</a> &middot; <a href="#jsonata-compatibility">Compatibility</a> &middot; <a href="#wasm">WASM Playground</a> &middot; <a href="CONTRIBUTING.md">Contributing</a> &middot; <a href="SECURITY.md">Security</a> </p>

---

What is gnata?

JSONata is a lightweight query and transformation language for JSON data — think "jq meets XPath with lambda functions." gnata brings the full JSONata 2.x specification to Go, with a production-grade streaming tier designed for evaluating thousands of expressions against millions of events per day with zero contention.

Features

• Full JSONata 2.x — path navigation, wildcards, descendants, predicates, sorting, grouping, lambdas, closures, higher-order functions, transforms, regex, and the complete 50+ function standard library.
• Two-tier evaluation — simple expressions use a zero-copy fast path (GJSON); complex expressions fall back to a full AST evaluator.
• Lock-free streaming — StreamEvaluator batches multiple expressions per event with schema-keyed plan caching. After warm-up, the hot path uses only atomic loads — no mutexes, no RWLocks, no channels.
• Zero allocations — simple field comparisons like user.email = "admin@co.com" evaluate with 0 heap allocations via GJSON zero-copy string views.
• Bounded memory — schema plan cache uses a FIFO ring-buffer with configurable capacity (WithMaxCachedSchemas), evicting the oldest entry on overflow.
• Context-aware — all evaluation methods accept context.Context for cancellation and timeouts. Long-running expressions check context at loop boundaries.
• Linear-time regex — uses Go's standard regexp (RE2 engine) for guaranteed linear-time matching with no timeouts or backtracking.
• 1,778 test cases — ported from the official jsonata-js test suite (0 failures, 0 skips).
• One dependency — tidwall/gjson for fast-path byte-level field extraction.
• ~13K lines of Go — complete implementation with no code generation.
• WASM support — compile to WebAssembly for an in-browser playground.

Quick Start

package main

import (
    "context"
    "fmt"
    "github.com/recolabs/gnata"
)

func main() {
    expr, _ := gnata.Compile(`Account.Order.Product.Price`)

    data := map[string]any{
        "Account": map[string]any{
            "Order": []any{
                map[string]any{"Product": map[string]any{"Price": 34.45}},
                map[string]any{"Product": map[string]any{"Price": 21.67}},
            },
        },
    }

    result, _ := expr.Eval(context.Background(), data)
    fmt.Println(result) // [34.45 21.67]
}

API Overview

gnata provides three evaluation tiers, each building on the previous:

Tier 1 — Eval(context.Context, any)

Evaluate against pre-parsed Go values. Pass a context for cancellation and timeouts.

expr, err := gnata.Compile(`$sum(orders.amount)`)
result, err := expr.Eval(ctx, data) // data is map[string]any

Tier 2 — EvalBytes(context.Context, json.RawMessage)

Evaluate directly against raw JSON bytes. For fast-path-eligible expressions, fields are extracted via GJSON with zero-copy — the entire document is never materialized.

expr, _ := gnata.Compile(`user.email = "admin@example.com"`)
result, _ := expr.EvalBytes(ctx, rawJSON) // rawJSON is json.RawMessage
fmt.Println(expr.IsFastPath())            // true — zero-copy evaluation

Tier 3 — StreamEvaluator

Batch-evaluate multiple compiled expressions against each event in a streaming pipeline. Schema-keyed plan caching deduplicates field extraction across expressions. Lock-free after warm-up.

se := gnata.NewStreamEvaluator(nil,
    gnata.WithPoolSize(500),
    gnata.WithMaxCachedSchemas(50000),
)

// Compile expressions (goroutine-safe)
indices := make([]int, len(rules))
for i, rule := range rules {
    indices[i], _ = se.Compile(rule.Expr)
}

// Hot path — millions of times, hundreds of goroutines
results, _ := se.EvalMany(ctx, eventBytes, schemaKey, indices)
for i, result := range results {
    if result != nil {
        handleMatch(indices[i], result)
    }
}

// Alternative: pre-decoded map input (avoids re-serialization)
results, _ = se.EvalMap(ctx, fieldMap, schemaKey, indices)

Custom Function Registration

Register domain-specific functions via WithCustomFunctions on StreamEvaluator or NewCustomEnv for standalone expressions. See Custom Functions below.

Streaming API

The StreamEvaluator is designed for high-throughput event processing where the same expressions are evaluated against millions of structurally similar events.

How It Works

Startup (once)
  Compile N expressions ──> Analyze AST: classify fast/full
  Configure stable routing: schemaKey -> exprIndices

Hot Path (millions/day, lock-free)
  Raw json.RawMessage event + schemaKey
    ├── BoundedCache lookup (atomic pointer read)
    │   ├── HIT  ──> Immutable GroupPlan
    │   └── MISS ──> Build plan (merge GJSON paths, atomic CAS store)
    ├── gjson.GetManyBytes: SINGLE scan for ALL expressions
    ├── Fast-path expressions: distribute extracted results (0 allocs)
    ├── Full-path expressions: selective unmarshal + AST eval
    └── results[]

Key Properties

• One JSON scan per event — all field paths needed by all expressions are merged into a single gjson.GetManyBytes call.
• Schema-keyed caching — the GroupPlan (merged paths, expression groupings, selective unmarshal targets) is computed once per schema key and reused immutably.
• Lock-free reads — BoundedCache publishes an atomic.Pointer snapshot on every write; reads scan the snapshot without acquiring a lock. Writes are serialised by a mutex.
• Selective unmarshal — full-path expressions unmarshal only the subtrees they need (e.g., just the items array from a 10KB event), not the entire document.
• Pre-decoded map input — EvalMap accepts map[string]json.RawMessage directly, skipping full-document serialization when the caller already has individually-encoded fields. Fast paths resolve top-level keys via O(1) map lookup.
• Dynamic mutation — Replace, Remove, and Reset methods allow modifying registered expressions at runtime with automatic cache invalidation.
• Observability — implement MetricsHook to receive per-evaluation callbacks for cache hits/misses, eval latency, fast-path usage, and errors.

Custom Functions

gnata supports registering user-defined functions that extend the standard JSONata library.

Defining Custom Functions

Custom functions implement the CustomFunc signature:

type CustomFunc func(args []any, focus any) (any, error)

Where args are the evaluated arguments passed by the JSONata expression and focus is the current context value.

Registration

Register custom functions via WithCustomFunctions when creating a StreamEvaluator:

customFuncs := map[string]gnata.CustomFunc{
    "md5": func(args []any, focus any) (any, error) {
        if len(args) == 0 || args[0] == nil {
            return nil, nil
        }
        h := md5.Sum([]byte(fmt.Sprint(args[0])))
        return fmt.Sprintf("%x", h), nil
    },
    "parseEpochSeconds": func(args []any, focus any) (any, error) {
        // Convert epoch seconds to ISO 8601
        f, ok := args[0].(float64)
        if !ok {
            return nil, nil
        }
        return time.Unix(int64(f), 0).UTC().Format(time.RFC3339), nil
    },
}

se := gnata.NewStreamEvaluator(nil,
    gnata.WithCustomFunctions(customFuncs),
    gnata.WithMaxCachedSchemas(10000),
)

// Expressions can now use $md5() and $parseEpochSeconds()
idx, _ := se.Compile(`$md5(user.email)`)
result, _ := se.EvalOne(ctx, eventJSON, "schema1", idx)

Standalone Expression Evaluation

For one-off evaluations with custom functions, create a custom environment and pass it to EvalWithCustomFuncs:

env := gnata.NewCustomEnv(customFuncs)
expr, _ := gnata.Compile(`$md5(payload.email)`)
result, _ := expr.EvalWithCustomFuncs(ctx, data, env)

The environment should be created once and reused across evaluations for best performance.

Metrics & Observability

The StreamEvaluator accepts an optional MetricsHook (via WithMetricsHook) for production telemetry. Implement the interface and wire it in — a nil hook (the default) adds zero overhead.

| Callback | Arguments | What to monitor | |---|---|---| | OnEval | exprIndex, fastPath, duration, err | Fast-path ratio; per-expression latency | | OnCacheHit | schemaKey | Cache hit rate — shou