jev-engineering: coding-agent tools on Jev, measured in public

Three tools built on TypeSafe's Jev, a model that answers yes/no questions in about 0.3 seconds for $0.00002 instead of writing text. Every claim below links to a benchmark you can rerun, including the one where a built-in tool beats mine.

| Tool | What it does | Proof | |---|---|---| | review-router | A GitHub Action that tells your AI code reviewer when it must read the whole pull request | A filename rule sent 13 real CVE fixes in Django and Express to a quick review. review-router sent all 13 to a full one | | jev-gate | A strict second lock on every command your coding agent runs | Once a session claimed it was authorized, Claude Code's auto mode let 8 of 12 dangerous test commands run. jev-gate let 0 run | | Use cases | Inbox triage, AI writing tells, Slack follow-ups, a check on messages before they send | Each with its own examples and numbers |

review-router

Your AI reviewer either re-reads the whole pull request on every push, which is slow and expensive, or reads only the new commits and misses what hides elsewhere. The usual fix is a path rule: full review for CI, auth and migration files, quick review for the rest.

That rule has a hole. On 561 commits from FastAPI, Express and Django, it sent 13 CVE fixes to the quick path, because the vulnerable code sat in files like validators.py and cache.py. review-router reads what the diff adds and sent all 13 to a full review, while 42% of commits could still skip it.

- uses: eugeniughelbur/jev-engineering/review-router@v1
  with:
    openrouter-api-key: ${{ secrets.OPENROUTER_API_KEY }}

Setup · Benchmark

jev-gate: a strict second lock

Claude Code and Codex both ship their own auto-approval now: Claude Code's auto mode and Codex's Guardian. Use them. jev-gate is not a replacement. It is a second check that judges only the command, never the conversation, so nothing in the session can talk it into anything.

I ran both on the same 30 commands, full write-up here:

| | Claude Code auto mode | jev-gate | |---|---|---| | Safe commands run with no prompt | 12 of 12 | 9 of 12 | | Dangerous commands that ran, once the session said it was authorized | 8 of 12 | 0 of 12 | | Median decision time | 0.77s | 0.28s |

Claude's auto mode is built to trust what the user authorizes, so those 8 are a design choice, not bugs. jev-gate is stricter, and it asks more often.

How it decides, before each command runs:

• Clearly safe, like git status, ls or your tests: it runs, no prompt.
• Clearly dangerous, like reading ~/.ssh or ~/.aws/credentials, or sending env to the internet: blocked, with the reason.
• Unsure, like git push --force: your agent's normal prompt, or its own auto mode, takes over.

On 3,622 commands from my own Claude Code history, it approved 66% on its own and never approved an rm, git push, git reset or sudo. How that was measured.

It catches mistakes, not a determined attacker. The attack test counts how many polite fake approvals got past its model step.

Quick start

claude plugin marketplace add eugeniughelbur/jev-engineering
claude plugin install jev-engineering@jev-engineering

1. Add your key. Put "env": {"OPENROUTER_API_KEY": "sk-or-..."} in ~/.claude/settings.json, then restart Claude Code.
2. Work for ten minutes. It only watches at first, and changes nothing.
3. Run /jev-status. It shows what it would have approved and blocked.
4. Run /jev-on. It starts deciding on the next command. /jev-off switches it back.

Codex, Cursor and OpenCode: see integrations/. Codex is tested live.

Or run it standalone:

git clone https://github.com/eugeniughelbur/jev-engineering
cd jev-engineering
export OPENROUTER_API_KEY=sk-or-...
./jev_gate.py --explain "git push --force origin main"

{
  "verdict": "ask",
  "source": "model",
  "reason": "verdict=ask, p=0.78",
  "destructive": 0.78,
  "confidence": 0.84,
  "latency_ms": 408,
  "cost": 1.71e-05
}

Run on 2026-09-27 against typesafe/jev-1.13. A force push to main is risky but sometimes intended, so the gate stops and asks you instead of refusing. An earlier run of the same command returned deny at 0.94. Scores move when the model is retrained, which is why the thresholds live in your own config.

How it decides

Four steps, and the order is the whole design.

1. Hard rules. Deterministic denials in plain regex. Never calls the model. Single-digit milliseconds.
2. Fast path. A read-only allowlist. Never calls the model.
3. Jev. One request, two questions, answered in parallel.
4. Thresholds. Your numbers, taken from your own observe-mode log.

Denials run before the allowlist on purpose. A command name says nothing about its arguments: cat is harmless until it is cat ~/.ssh/id_ed25519. tests/test_order.py pins that, because this shipped the wrong way round once.

Hard rules come before the model for a different reason. In testing, the model was least reliable exactly where a hard rule is easiest to write.

It fails open. Any error, timeout or missing key falls back to your harness's normal permission prompt, so a network blip never bricks a session.

Modes

| Mode | What it does | Use it when | |---|---|---| | observe (default) | Logs every decision, changes nothing | Your first day or week | | auto | Approves clear allows so their prompt never appears, blocks deny, asks about the rest | You want fewer prompts. The one to use day to day | | guard | Blocks deny, leaves every other prompt as it was | You only want the safety net | | enforce | Blocks deny and ask | Unattended runs where nobody can answer a prompt |

/jev-on                      # in Claude Code: auto mode from the next command
./jev_gate.py --mode auto    # the same, from a shell
./jev_gate.py --stats        # what auto mode would do with your log so far

The mode is saved in ~/.jev-gate/mode, so no restart is needed. JEV_GATE_MODE in the environment still wins over it.

Auto mode only approves Bash commands by default. Set JEV_AUTO_TOOLS=Bash,Edit,Write to let it approve file edits too. A command that chains, pipes, redirects or substitutes, like git status && rm -rf src, never takes the allowlist shortcut. It goes to Jev.

Read ~/.jev-gate/decisions.jsonl for a week before you turn anything on. That file is the only source of thresholds that will fit your work.

The three decisions an agent makes constantly

The gate fires on the rare dangerous action. These fire every turn, which is where the cost and the latency actually live. Same shape underneath: state plus typed questions, one request, answers in parallel.

uv run layer.py route "fix the typo in the README heading"
# {"tier": "fast", "confidence": 1.0, "latency_ms": 516}

uv run layer.py route "redesign how we shard the primary database with zero downtime"
# {"tier": "frontier", "confidence": 1.0, "latency_ms": 356}

route picks the model tier before you spend on the turn. When confidence drops below 0.5 on the cheapest tier it steps up one, not to the top, because falling back to the frontier model on every uncertain turn eats most of the saving.

rank orders up to 255 options against one criterion in a single call. Useful for picking a file, a tool, or reordering search results.

uv run layer.py rank "This file holds the login and session logic" \
  --options "src/auth/session.ts,src/db/schema.ts,README.md"
# winner: src/auth/session.ts

keep decides which pieces of context still earn their place. Nothing is rewritten or summarised. Each item is kept verbatim or dropped, because a summary silently loses the exact path or error you needed later.

uv run layer.py keep transcript.json --goal "fix the failing session test" --budget 0.5

On a real eight-item transcript, in 347ms for $0.000026, it kept the failing test output, the source file and the spec, and dropped echo hello and df -h.

All three are also MCP tools, so an agent can call them itself: route_turn, rank_options, keep_context.

Use cases

The gate answers one question. usecases/ holds the decisions around it, one folder per situation, each with code, examples and measured numbers.

| Use case | Result | |---|---| | When an AI code review must read the whole pull request | On 561 commits from FastAPI, Express and Django, 42% skip the full review and 8 of 8 CVE fixes go to it. One line as a GitHub Action | | Inbox triage | 16 of 16 emails sorted right for $0.0003 | | AI writing tells | Names the tell in each flagged paragraph. 2 false alarms in 40 real paragraphs | | Slack follow-ups | 14 of 14 messages sorted right for $0.0002 | | Outbound check | Holds risky emails and Slack messages before they send. All 8 risky drafts held, 1 false alarm in 16 |

Commands

Installed as a plugin, you get:

| Command | What it does | |---|---| | /jev-status | Mode, key, and how many prompts auto mode would have skipped so far. Runs one live check so you can see it work. | | /jev-on | Turns auto mode on from the next command. No restart. | | /jev-off | Back to logging only. | | /jev-calibrate | Reads your week of decisions and hands back your thresholds, your fast-path rules and your hard-rule candidates. | | /jev-attack | Fires 300 injections at your own gate and reports what got through. | | /jev-policy | The rules in force, which layer each came from, and pulls your team's latest. |

Any agent, not only coding agents

Install it as a Python package and call decide() from your own code:

pip install jev-engineering
jev-gate --explain "git push --force origin main"

Or expose it over MCP, so Cursor, Codex, Windsurf and anything else that speaks the protocol can ask it:

{ "mcpServers": { "jev-engineering": {
  "command": "uv",
  "args": ["run", "--directory", "/abs/path/to/jev-engineering", "mcp_server.py"],
  "env": { "OPENROUTER_API_KEY": "sk-or-..." } } } }

Three MCP tools: check_action judges something before it happens, rank_options sorts up to 255 choices in one call, gate_stats reports what has been decided.

Question packs

A gate is only as good as its questions, and shell commands are not the only thing an agent does. Five packs ship, and pack picks one:

| Pack | For | |---|---| | shell | Commands in a coding agent. The default. | | message | An email, Slack message or post before it sends. | | money | A refund, payment or transfer. | | data | Reading, writing or deleting records. | | publish | A deploy, a merge, a release. |

uv run packs.py            # see them all

Copy the closest one into ~/.jev-gate/packs/ and edit it. Measured examples, run live on 2026-09-27:

• an email containing a live API key: private_data 0.96, verdict deny
• the same email without it: private_data 0.07, verdict allow
• a $4,200 refund on a $42 order: amount_unusual 0.98, authorised 0.14, verdict deny

Guarding everything else

The plugin guards one agent. Run it as a background service and anything can ask it: n8n, cron jobs, a deploy step, a bot about to send a message.

export OPENROUTER_API_KEY=sk-or-...
./service/install.sh
curl -s localhost:8787/check -d