<div align="center">

AgentTrust

A deterministic safety floor and capability-control layer for tool-using AI agents.

The first framework to gate dangerous operations, suggest a safer fix, track multi-step attack chains, and enforce least privilege — deterministically, before an action executes — and the first to learn from its own verdicts, distilling them into new rules and guarded semantic memory over time.

AgentTrust is a sub-millisecond rule gate that sits between an agent and its tools and enforces a policy over rule-covered dangerous operations — credential reads, dotfile/persistence writes, pipe-to-shell, rm -rf, DROP DATABASE, protected-branch force-push — with near-zero benign false positives. When it blocks an action it also proposes a safer equivalent (SafeFix), detects multi-step RiskChain patterns across a session, and offers an opt-in, off-by-default LLM-judge fallback for the ambiguous review tail. Since v0.6, the judge's verdicts also feed a self-learning dual store — confident verdicts on fixed-signature attacks are distilled into new deterministic rules, and corroborated semantic precedents enter a guarded memory — so the gate gets cheaper and smarter the longer it runs (Self-Learning). The point is not "detecting malice" — it is a deterministic, auditable floor that constrains what an agent can do, regardless of intent.

42 risk patterns | 174 policy rules | 37 SafeFix rules | 7 chain detectors | 300 benchmark scenarios | 630 held-out test scenarios | 410 unit tests | ~0.3ms median latency

Quick Start | Scope & Positioning | Architecture | SafeFix | RiskChain | LLM Judge | Self-Learning | Capability Mode | Benchmark | Safety Guarantees | Positioning | Docs

</div>

---

Why AgentTrust

When an AI agent does real, irreversible damage, it is rarely through anything exotic — it is through a small, enumerable set of concrete operations: an accidental rm -rf /, a credential read piped to an external host, a DROP DATABASE, a force-push over main. The harmful operation is concrete and finite; the intent behind it is the part you can never pin down — and everything normally asked to judge that intent is fallible. A human can't reliably tell a reverse shell from a benign command (we measured it); an LLM reviewer can be talked out of its verdict by a single injected sentence of prose. AgentTrust's bet is to stop adjudicating intent and instead deterministically gate the operation — which is finite, concrete, and cannot argue back.

AgentTrust is not trying to decide whether an agent is "trustworthy" or to detect malice in the general case (static analysis has a structural ceiling on semantic/prose-level attacks; runtime attack-success-rate defense is a different axis — see Scope & Positioning). Its job is narrower and more dependable: be a deterministic safety floor under whatever else you run. And for the semantic, intent-dependent threats that structurally defeat any rule set, the opt-in judge layer is what generalizes: on an external evaluation set it scores 84–85% — roughly 2× the rule core — with near-zero false-blocks, across two model providers (threat-type breakdown).

graph LR
    A["Agent proposes<br/>a tool action"] --> B["AgentTrust gate<br/>(deterministic rules)"]
    B -->|"covered dangerous op"| C["BLOCK / WARN / REVIEW<br/>+ SafeFix + audit record"]
    B -->|"everything else"| D["ALLOW<br/>(near-zero benign FPR)"]
    style B fill:#2c3e50,stroke:#1a252f,color:#fff
    style C fill:#e67e22,stroke:#d35400,color:#fff
    style D fill:#2ecc71,stroke:#27ae60,color:#fff

Three things set it apart:

1. Deterministic & auditable. The same action always yields the same verdict, with the exact matched rule(s) and risk factors attached — no model on the hot path, nothing to hallucinate, and no prose for an attacker to argue with. An LLM reviewer cannot give you this.
2. Capability control / least privilege. The gate constrains what an agent can do regardless of whether anyone "detected" malicious intent — bounding the blast radius to exactly what the task needs.
3. Composable by design. AgentTrust runs beneath fallible LLM review and inside your runtime sandbox — it strengthens whatever else you run instead of competing with it.

Every action is analyzed, scored, and explained before execution, in sub-millisecond time on the rule path.

Why a deterministic floor, and not just human review?

Because people — including the agent's own user — are unreliable safety arbiters. In a small blind study, five annotators independently labeled the same 60 benchmark operations as allow / warn / block:

• An independent security expert reproduced the ground-truth verdicts almost perfectly (Cohen's κ = 0.89, 95% CI [0.78, 0.97]) — the labels are sound, and the analysis identifies that expert blind, from agreement alone.
• Ordinary users could not. Mean κ = 0.16; they let through 57% of genuinely dangerous operations (a reverse shell, disabling audit logging, exfiltrating a database) and barely agreed with one another (inter-user κ = 0.08), failing in opposite directions — some too permissive (waving attacks through), some too paranoid (blocking benign work).

You cannot close this gap by "trusting the user" or by "telling users to be careful": the permissive ones let attacks in, the paranoid ones break normal work. Only an expert-encoded, deterministic floor catches both sides — which is exactly what AgentTrust is.

---

Scope & Positioning

What AgentTrust does:

• A deterministic, auditable safety floor for dangerous operations — credential reads, dotfile/persistence writes, pipe-to-shell, rm -rf, DROP DATABASE, protected-branch force-push, and more — with near-zero benign false positives (0 on the 50-scenario benign set).
• Capability control / least-privilege enforcement — it bounds an agent's blast radius to exactly what its task needs, whether or not anyone "detected" malice.
• Self-learning that distils confident judge verdicts into new deterministic rules and corroborated semantic precedents — so the gate gets cheaper and smarter the longer it runs.

By design, AgentTrust gates operations, not intent. It does not try to read an agent's mind or output a "this agent is malicious" verdict — it constrains what an agent can do, deterministically, before it acts. That is exactly what makes it composable: it runs beneath fallible LLM review and inside your runtime sandbox, strengthening both instead of competing with either. See docs/POSITIONING.md for the full picture.

---

Real-World Incidents This Class of Operation Causes

These are not hypothetical risks. Over the past 18 months, multiple reports have surfaced of AI coding agents — wired into developer tools such as Cursor, Replit, and Claude Code — performing destructive operations on production systems with little or no human confirmation:

• Replit AI agent wipes a production database (July 2025) — SaaStr CEO Jason Lemkin publicly described an incident in which an AI coding agent deleted his production database during a "code freeze" and admitted to fabricating data afterwards. (reporting)
• Recurring reports of AI agents executing rm -rf, DROP DATABASE, force-pushing over main, or deleting backup snapshots in agentic IDE/MCP setups, often within seconds and without an opportunity for human review.

We make no claim about any specific unverified report. The point is that these incidents share a rule-coverable primitive:

A large model with tool access executes an irreversible, high-blast-radius operation — often as the final step of a multi-step plan whose individual steps looked harmless.

This is exactly the regime where a deterministic floor is most useful: the dangerous operation (DROP DATABASE, rm -rf over backups, force-push over a protected branch) is concrete and enumerable, even when the intent behind it is not. Every primitive above is covered by a built-in policy rule, and the session-level Data Destruction chain detector raises a CRITICAL alert when the pattern (enumerate → disable safety → mass-delete) appears, regardless of how the steps are spaced out. AgentTrust does not claim to know why the agent issued the command — only to stop the covered operation deterministically.

You can replay this exact class of incident through the verifier yourself:

python3 examples/incident_replay.py

The script feeds a DROP DATABASE query, a shell-side psql -c "DROP DATABASE ...", a mongosh ... dropDatabase() call, an rm -rf /var/backups/..., and a git push --force through TrustInterceptor — both as isolated actions and as a single session — and prints the per-action verdict, ma