AVO — Agentic Variation Operators

An open reproduction of AVO: Agentic Variation Operators for Autonomous Evolutionary Search (Chen, Ye, Xu et al., NVIDIA, 2026), runnable on a laptop.

Classical evolutionary search, and the LLM-augmented systems that followed it, decompose the variation operator into a fixed pipeline:

Vary(P_t) = Generate(Sample(P_t))

The framework samples parents; the model produces one candidate from them. AVO replaces that whole decomposition with a single autonomous agent run:

Vary(P_t) = Agent(P_t, K, f)

The agent sees the full lineage P_t, a domain knowledge base K, and the scoring function f — and decides for itself what to read, what to change, and when to measure. It stops being a candidate generator and becomes the variation operator.

This repo implements that framework, plus the surrounding machinery the paper describes: a git-backed lineage, a correctness-gated score vector, the matches-or-improves commit policy, a supervisor that intervenes on stagnation, and trajectory plots. Two optimisation targets ship with it.

---

The part that matters: it runs on the session you already have

The default driver does not spawn an agent and does not call an API. It hands the variation prompt to the Claude Code session you are already talking to, and that session does the work. Nothing extra is billed, no ANTHROPIC_API_KEY is needed, and the agent doing the optimising is a real general-purpose coding agent — which is exactly what the paper used.

Unattended mode (spawn an agent per step and let it run for days, like the paper's 7-day experiment) is available too, and is opt-in precisely because it spends quota.

---

Install

git clone https://github.com/gatordevin/avo
cd avo
pip install -e ".[all]"     # or: pip install -e .  for the core only
avo doctor

On a system with an externally-managed Python (Homebrew, most Linux distros), use a virtualenv — the --system-site-packages flag reuses a NumPy and Matplotlib you already have:

python3 -m venv --system-site-packages .venv
.venv/bin/pip install -e ".[all]"
.venv/bin/avo doctor

Requirements: Python 3.10+, git, and a C compiler if you want the attention_c target. numpy is needed by the bundled targets, matplotlib for plots. The core framework depends only on PyYAML.

---

Quickstart — drive it from the agent you already have

Full protocol, including Codex and plain-CLI use, in docs/DRIVING.md.

Claude Code

Register the MCP server once, at user scope so it is available in every folder:

claude mcp add avo -s user -- python3 -m avo.mcp_server
# from a virtualenv, point at its interpreter:
claude mcp add avo -s user -- /path/to/avo/.venv/bin/python -m avo.mcp_server

claude mcp list should show avo — ✔ Connected. Optionally install the bundled skill so /avo works anywhere:

cp -r .claude/skills/avo ~/.claude/skills/avo

Then, in a Claude Code session in any directory:

Use the avo tools to evolve the game2048 target for 10 steps. Call avo_start_run, then loop: avo_next_step, do the work it asks for, avo_evaluate until you're happy, then avo_submit. If it reports a stall, call avo_supervisor_brief, answer it, and file it with avo_record_supervisor.

The eleven tools are the whole loop:

| tool | what it does | |---|---| | avo_start_run | seed x_0, score it, measure baselines, open the lineage | | avo_next_step | the variation prompt: P_t, the index of K, the contract for f | | avo_evaluate | run f on the work tree — free, call it as often as you like | | avo_submit | end the step: score, then commit or revert per the policy | | avo_revert | abandon an experiment without spending the step | | avo_status / avo_lineage | where the run is | | avo_supervisor_brief / avo_record_supervisor | the stagnation intervention | | avo_plot | render the trajectory | | avo_list_targets | what can be evolved |

Codex

Codex CLI speaks MCP and reads AGENTS.md, so both halves work:

codex mcp add avo -- python3 -m avo.mcp_server

AGENTS.md at the repo root documents the loop and the rules that keep a run honest; Codex picks it up automatically when working in this directory.

Without MCP

Every tool has a CLI twin, so a plain shell works just as well — this is the most portable option and works with any agent, or by hand:

avo start --target game2048          # seeds x0 and prints the first prompt
# ... edit runs/<id>/work/, run runs/<id>/avo-eval as often as you like ...
avo submit -m "expectimax depth 2 with a positional weight matrix"
avo prompt                           # the next step's prompt
avo status
avo plot -o trajectory.png

---

Worked runs

Two complete runs ship with the repo, both driven in session mode by a Claude Code session, both including their dead ends.

attention_decode — beating the vendor kernel

examples/attention-decode-run/ evolves the decode step of attention: one query token against a long KV cache, the computation an LLM runs for every generated token. Scored against mx.fast.scaled_dot_product_attention — Apple's own fused Metal kernel.

0.05 → 1.14× MLX in three steps. This is the one where the evolved kernel actually beats the vendor implementation, and the interesting part is how:

• Step 1 was implementation — split-K flash-decoding took the kernel from 1.6 GB/s to 106 GB/s, about 95% of the machine's streaming limit. That reached 0.95× MLX and exhausted the lever: you cannot read bytes faster than the memory controller delivers them.
• Step 2 was mathematics. The target's gate is an output-error budget rather than exact equality, so the search could change the computation. Measurement showed 99.9% of the softmax mass sits in ~11% of keys, so the kernel now scores every key but reads V only above a threshold derived so the discarded mass is provably under 0.3%. That crossed 1.0, spending 2% of the error budget.

The lesson generalises: once a bandwidth-bound kernel is at the roofline, the only remaining lever is to read fewer bytes, and that is an algorithmic change.

Read the full write-up →

attention_c — the paper's own domain

examples/attention-c-run/ evolves a forward attention kernel in C, reaching 2.2× a straightforward NumPy/BLAS implementation and close to the NEON roofline. Note the honest framing: that baseline is not a tuned attention library, and this kernel is slower than torch's CPU SDPA and MLX — Apple's AMX matrix units are unreachable from portable C. The write-up gives the full comparison.

Three findings from it are worth the click:

• The paper's own algorithm was the wrong answer here. A FlashAttention-style tiled kernel with a streaming online softmax measured worse, twice. At these sizes a whole head fits in L2, so blocking for locality buys nothing while the per-block rescale is pure added work. The cost is arithmetic, not memory.
• -ffast-math silently breaks the standard fast-exp, by algebraically cancelling the add-magic-constant rounding trick it depends on. The correctness gate caught it on an N=3 shape; the throughput number never would have.
• The run forced a target fix. Scoring raw GFLOP/s on a laptop doing other work is not a measurement — identical code ranged 44–76 GFLOP/s in twenty minutes. eval.py now times a NumPy/BLAS reference in the same process, interleaved with the candidate, and scores the ratio.

Read the full write-up →

game2048 — evolving a game-playing policy

examples/game2048-run/ is a complete 8-step run of the game2048 target, driven in session mode by a Claude Code session. The directory holds the unedited output: the evolved policy, the operator's working notes, the full trajectory, the screening tools it built, and its dead ends.

876 → 43 826 — 50× the seed, 14× the strongest baseline. Games reaching 2048: 0% → 77%. Best tile: 512 → 8192. Apple M5, single-threaded, standard library only.

Improvement arrives in discrete jumps separated by plateaus, matching the paper's Figure 5. The two flat versions are pure throughput work that bought the budget the next step spent — the same role the paper's v19→v20 branchless-rescale change plays.

The largest single gain (+50.5%) was not an optimisation. The benchmark scores accumulated game points; the heuristic only measured how survivable a board looked, so nothing in the search knew that merging two 256s banks 512 points. Four steps of throughput work were worth +27% combined; one step of checking what was actually being optimised was worth +50%.

Read the full write-up →

What ships with it

game2048 — evolve a game-playing policy

Evolve agent.py into the strongest 2048 player you can, under a hard thinking-time budget. Scored as the geometric mean of mean game score across four banks of twelve deterministic seeds. Blowing the 120 s budget scores zero, not "slightly less" — so search depth, evaluation-function cost, and pruning all trade against each other, and that trade-off is the problem.

Measured on an Apple M5:

| policy | score | |---|---| | seed x_0 (first legal move) | 876 | | random baseline | 1 076 | | corner heuristic baseline | 2 565 | | greedy one-ply baseline | 3 132 |

Strong expectimax players score in the tens of thousands. The worked run above reached 43 826.

attention_c — evolve a kernel, the paper's own domain

Evolve a single-precision forward attention kernel in C: O = softmax(QKᵀ/√D)V, causal and non-causal, D = 64. Gated on agreement with a float64 reference over eighteen shapes — including prime and off-by-one sequence lengths, so a kernel that mishandles its tail fails rather than quietly scoring well.

Scored as speedup over a NumPy/BLAS reference timed in the same process, geometric mean across four sequence lengths × two masking modes. 1.0 is parity with the library. Scoring a ratio rather than raw GFLOP/s makes the benchmark immune to whatever else the machine is doing — absolute throughput on a shared laptop moves by more than most optimisations are worth.

| kernel | score | |---|---| | seed x_0 (naive, materialises the full N×N score matrix) | 0.19× | | NumPy/BLAS baseline — the "cuDNN" of this setup | 1.00× | | evolved in 3 steps (write-up) | 2.11× |

The knowledge base covers the online-softmax formulation, tiling and block-size selection, CPU vectorisation, threading, and how to interrogate the host machine rather than assuming an ISA. Beating BLAS needs most of them.

---

How it works

The run directory

runs/<run-id>/
  work/              the candidate x_t — a standalone git repo whose history IS the lineage
    .avo/scores.jsonl    every committed version's full score vector
  kb/                the knowledge base K, copied in so paths are stable
  avo-eval           f, as a zero-argument shim the agent can call at will
  NOTES.md           scratch space that survives across steps
  trajectory.jsonl   every step, accepted or rejected
  rejected/          the diff of each rejected candidate, kept for the record
  logs/              evaluator and agent logs

Making the lineage a git repo means the agent inspects P_t with tools it already knows — git log, git show v7:attention.c, git diff v6 v7 — instead of a bespoke API. Each accepted version is a commit tagged vN whose message carries the score vector.

The commit policy

Paper §3.2: a candidate is committed only if it pas