TokenEconomizer: Expe…[redacted] Black-Box Cost Control for Coding Agents

<p align="center"> Aaron J. Li¹, Weijia Xu², Chandan Singh², Zixiao Chen³, Jeevana Priya Inala², Charlotte Siska³, <br> Acar Tamersoy³, Bin Yu¹, Ion Stoica¹, Michel Galley², Jianfeng Gao² </p> <p align="center"> ¹UC Berkeley &nbsp; ²Microsoft Research &nbsp; ³Microsoft Security </p> <p align="center"> <a href="https://arxiv.org/abs/2610.22732"><img src="https://img.shields.io/badge/arXiv-2610.22732-b31b1b.svg" alt="arXiv"></a> <a href="https://tokeneconomizer.github.io"><img src="https://img.shields.io/badge/project-page-2f6fb0.svg" alt="Project page"></a> <a href="https://x.com/aaron_jli/status/2114797313613053952"><img src="https://img.shields.io/badge/X-post-000000.svg?logo=x&logoColor=white" alt="X post"></a> <img src="https://img.shields.io/badge/python-3.10%2B-3776ab.svg" alt="Python 3.10+"> <img src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" alt="Apache 2.0 license"> </p> <p align="center"> Our measurement on SWE-Bench tasks: <b>18.4–30.9% lower mean cost</b> across five agent–model combinations, cheaper in <b>all 30</b> repository × combination cells, and Pass@1 up in <b>4 of 5</b>. <a href="#results">Results</a> </p> <p align="center"> <picture> <source media="(prefers-reduced-motion: reduce)" srcset="docs/assets/teaser.png"> <img src="docs/assets/teaser.gif" width="100%" alt="An animation that opens on a title card: TokenEconomizer, experience-driven black-box cost control for coding agents, with its three mechanisms (micro-skills, tool pruning, failure-loop intervention), the agents it works with (Claude Code, Codex, GitHub Copilot CLI, Pi) and the UC Berkeley and Microsoft logos. Then a schematic of six sessions in one repository, each a bar of what it cost: the work itself plus three kinds of waste, namely re-learning the repo, tools never used, and getting stuck in loops. Each of the first three sessions teaches TokenEconomizer one thing, in order: 1 micro-skills remember how the repo works (run tests with python -m pytest), 2 tool pruning skips tools never used here (Read and Edit kept, Notebook and Browser left out), and 3 failure-loop intervention spots loops and steps in (the same error three times meets a nudge to try another way). In the session after each lesson that waste shrinks, so the bars step down below a dashed line marking the cost without TokenEconomizer. It ends on results measured on SWE-Bench tasks: 18.4–30.9% lower mean cost across five agent–model combinations, cheaper than default in 30 of 30 repository × combination cells, and Pass@1 up in 4 of 5; per combination, cost 23.6% lower and Pass@1 +1.3 for Claude Code with Sonnet 5, 30.9% and −1.0 with Opus 4.8, 24.7% and +0.9 for Codex with GPT-5.6 Terra, 26.1% and +1.6 with GPT-5.6 Sol, and 18.4% and +2.6 for Copilot CLI with Sonnet 5."> </picture> </p>

TokenEconomizer makes coding agents cheaper to run in the repositories you work in often, without changing the agent itself. It learns from the sessions an agent has already run in a repository, and uses that experience to cut waste in the sessions that follow. It has three mechanisms:

• Micro-skills: remember how the repository works. Sessions keep rediscovering the same facts: a build step that must run first, the spelling of a command that works here (python -m pytest where pytest is missing, npx tsc where tsc is), an environment variable a command needs, where generated files live. Micro-skills learns such facts, in any language, from two sources: failures that repeat across sessions, and a model that reviews the recent sessions every k sessions (k = 3) and proposes facts. A fact is stated at the start of each session once three sessions have confirmed it, and dropped when later sessions contradict it.
• Tool pruning: stop loading tools that go unused. Every loaded tool's definition is sent with every request. A tool that goes unused over p prompts spread across at least m sessions, or over s sessions of any length (p = 10, m = 2, s = 5), is left out of the next sessions through the agent's own settings. One command brings it back.
• Failure-loop intervention (the loop guard): step in when a turn is stuck. When a turn (the agent's work on one prompt) keeps hitting the same failures, a judge model checks whether it is stuck. If the judge is at least c sure (c = 0.7), the agent gets a redirect (a short note, added to its context, to try another approach), at most r times per turn (r = 1). If the turn is still stuck after that, TokenEconomizer tells you how to stop it; it never stops a turn itself.

TokenEconomizer is a plugin for Claude Code, Codex, the GitHub Copilot CLI and Pi, with experimental support for opencode. Each italic letter above is a setting, shown with its default (Settings), and each mechanism can be turned on or off. The plugin announces each action in one line, and if any part of it fails, your session carries on as if it were not installed. Its model calls (the fact proposer, and the judge with the redirect it writes) run through your own agent, on the same plan or login, and their cost is included in its reports. What it learns stays on your machine, in ~/.tokeneconomizer.

Results

The paper ran five agent–model combinations on the same 277 SWE-Bench tasks from six repositories. Each task ran as its own session, and each repository's tasks ran in order, starting with nothing learned. Costs include every model call TokenEconomizer made. Within each repository and combination, cost fell by 5–40%. The savings held on Web-Bench (16–21% lower) and on two security-auditing benchmarks where every task is in a new repository (6–20% lower). The project page has interactive charts, per-repository numbers and nine annotated runs, and experiments/ reruns the SWE-Bench and Web-Bench experiments.

How it works

<p align="center"><img src="docs/assets/framework_overview.png" width="100%" alt="TokenEconomizer: the plugin keeps what it knows about a repository and applies micro-skills, tool pruning and failure-loop intervention to your unchanged coding agent; each finished session's log updates all three"></p>

Coding agents are built to finish tasks, not to keep them cheap. Every turn pays again for the agent's instructions, every loaded tool, and the conversation so far. Developers use the same agent on the same repository again and again, so much of that cost repeats: the same facts rediscovered, the same unused tools sent, the same kinds of failing loops. TokenEconomizer leaves the agent's model, weights and code alone and learns across a repository's sessions instead of treating each one on its own, so whatever the plugin spends is paid back over the sessions that follow. Four principles guide the mechanisms:

• Work through what the agent already reads. Facts enter the context at session start, and tools are removed through the agent's own settings, so nothing depends on the agent learning a new tool.
• Count cost as it is billed. Most tokens in a session are cached re-reads, so cost is computed with cache pricing, and it includes the plugin's own model calls.
• Protect task success. Facts form only from repeated evidence, tools are removed only after going unused, and the loop guard leaves alone a turn that is still making progress.
• Keep every action reversible. A fact that later sessions contradict is dropped, removed tools can be given back, and the loop guard only redirects and tells you: stopping a turn is your call.

Quick start

You need one of these agents, signed in any way it supports, and Python 3.10 or newer on PATH. The plugin needs no API key of its own and never handles credentials, so any plan or login works.

Install

Run these commands in the folder where you want to keep the clone. Claude Code, Codex, the Copilot CLI and opencode run the plugin from the clone, so leave it in place. Then start the agent in one of your repositories.

Claude Code

git clone https://github.com/microsoft/TokenEconomizer.git
claude plugin marketplace add "$PWD/TokenEconomizer"
claude plugin install tokeneconomizer@tokeneconomizer

Start claude, trust the folder (Yes, I trust this folder), and run /tokeneconomizer:status, allowing its script if asked. Once per machine, and again after Claude Code updates, run /tokeneconomizer:toolprune refresh-catalog.

Codex

git clone https://github.com/microsoft/TokenEconomizer.git
codex plugin marketplace add "$PWD/TokenEconomizer"
codex plugin add tokeneconomizer@tokeneconomizer

Start codex, choose Trust and continue for the folder, then at Hooks need review choose Trust all and continue. The hooks take effect from your next Codex session, so quit (/exit), start codex again, and check with $tokeneconomizer:status.

Copilot CLI

git clone https://github.com/microsoft/TokenEconomizer.git
copilot plugin marketplace add "$PWD/TokenEconomizer"
copilot plugin install tokeneconomizer@tokeneconomizer

Start copilot, choose Yes, and remember this folder for future sessions, and run /tokeneconomizer:status, choosing Yes to run its script. Once per machine, and again after Copilot updates, run /tokeneconomizer:toolprune refresh-catalog.

Pi

pi install git:github.com/microsoft/TokenEconomizer

Start pi and run /te status.

opencode (experimental)

git clone https://github.com/microsoft/TokenEconomizer.git
mkdir -p ~/.config/opencode/plugins
ln -s "$PWD/TokenEconomizer/plugin/opencode/tokeneconomizer.ts" ~/.config/opencode/plugins/

Start opencode and send a prompt, then check in the repository with ~/.tokeneconomizer/bin/te --host opencode status. opencode has no plugin commands, so the plugin's commands run from a terminal.

If the install worked, the status command prints a short status block. refresh-catalog records every tool a session loads (without a model call), so that tool pruning can consider all of them. Keep the marketplace path absolute, as above: the Copilot CLI reads a relative path such as ./TokenEconomizer as the name of a GitHub repository.

To update:

| Agent | Update | |---|---| | Claude Code, Copilot CLI, opencode | git pull in the clone | | Codex | git pull in the clone, then the codex plugin add line again | | Pi | pi update --extensions |

plugin/README.md walks through each agent in more detail, including uninstalling, using Copilot with your own model key, and troubleshooting.

Supported versions

| Agent | Supported | Older versions | |---|---|---| | Claude Code | 2.1.300 to 2.1.310 | not supported | | Codex | 0.153.1 to 0.164.0 | not supported | | Copilot CLI | 1.0.83 to 1.0.102 | not supported | | Pi | 1.2.0 to 1.3.0 | 0.99.x: load the extension by path (below) | | opencode (experimental) | 1.19.5 and 2.2.0 | not tested |

The commands above were tested on the oldest and newest version in each range; newer versions have not been tested yet.

• Codex before 0.141.0 does not run plugin hooks at all.
• Pi 0.99.x: instead of pi install, clone the repository and start each session with pi --extension /path/to/TokenEconomizer/plugin/pi/tokeneconomizer.ts, using your clone's full path; git pull in the clone updates it.
• opencode support is experimental: it was tested on 1.19.5 and 2.2.0 only, and not in the 2.x terminal UI.

Everyday commands

Work as usual; the plugin needs nothing from you. To see or change what it does, use its commands. They have the same names on ever