SkillAgentSearch skills...

trimtok

Cut token usage in Claude Code, Cursor, Codex & Gemini: deterministic in-process context compression for AI coding agents. Lossless and retrievable (#ref), AST code skeletons, TOON JSON tables, stdio MCP. No LLM, no network, MIT.

Install / Use

claude mcp add Stxyu-p -- npx -y github:Stxyu-p/trimtok

If the server publishes to npm under a different name, use that package instead — check the repo README.

About this skill
🔌

MCP Server

Model Context Protocol server

Quality Score

83/100

Supported Platforms

Claude Code
Claude Desktop
Cursor
Gemini CLI
OpenAI Codex

Our assessment of trimtok

trimtok scores 83/100 on our quality scale, 661st of 947 AI & Machine Learning skills we index.

Its MCP Server is 39 KB long, well organised into 66 sections with 25 code examples: a thorough specification that gives an agent plenty to work with.

It has 3 GitHub stars, so there is little community track record yet; judge it on its content.

Substance
30/30
Structure
20/20
Description
15/15
Adoption
3/20
Freshness
15/15

Maintenance, license and trust

  • The repository was last updated yesterday, so trimtok is actively maintained.
  • It is released under the MIT license, a permissive license that allows use, modification and commercial use with attribution.
  • Its trust signals score 92/100, with 1 caution from licensing, adoption, age or documentation. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.

Safety scan

No issues found

Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful.

AI review by kimi-k2.7-code on 2026-11-15. Automated pattern scan on 2026-11-15. It catches known dangerous patterns, not every risk — read a skill before letting an agent act on it.

trimtok compared with similar skills

All 4 of these similar skills score higher than trimtok; compare them before choosing.

SkillScoreStarsUpdatedFormat
trimtok (this skill)by Stxyu-p8331d agoMCP Server
claude-memby thedotmack100117.6ktodayCLAUDE.md
Agent-Reachby Panniantong100112.3k8d agoCLAUDE.md
Understand-Anythingby Egonex-AI10092.8k1d agoCLAUDE.md
rufloby ruvnet10081.9ktodayCLAUDE.md

Frequently asked questions

How do I install trimtok?
Run claude mcp add Stxyu-p -- npx -y github:Stxyu-p/trimtok. The install tabs above show the steps for each supported agent.
Which AI agents does trimtok work with?
It is written for Claude Code, Claude Desktop, Cursor, Gemini CLI and OpenAI Codex, as a MCP Server file. Other agents that read the same format can often use it too.
Is trimtok safe to use?
Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful. It is MIT-licensed and scores 92/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
Is trimtok still maintained?
The repository was last updated yesterday, so trimtok is actively maintained.
<h1 align="center">⚡ trimtok</h1> <p align="center"> <strong>Universal, Local, Deterministic Context Compressor &amp; Content Intake Shield for Autonomous AI Coding Agents</strong> </p> <p align="center"> <a href="https://pypi.org/project/trimtok/"><img src="https://img.shields.io/pypi/v/trimtok?style=flat-square" alt="PyPI" /></a> <a href="https://www.python.org/"><img src="https://img.shields.io/badge/python-3.10+-blue?style=flat-square" alt="Python" /></a> <a href="#how-it-works"><img src="https://img.shields.io/badge/latency-sub--ms_hook_%7C_200ms_guard-10b981?style=flat-square" alt="Latency" /></a> <a href="#7-ccr-recovery"><img src="https://img.shields.io/badge/storage-SQLite_WAL_+_Zstd-07405E?style=flat-square" alt="Storage" /></a> <a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-slate?style=flat-square" alt="License" /></a> </p> <p align="center"> <img src="assets/before-after.svg" width="100%" alt="trimtok - Before vs After Context Shield" /> </p>

<a id="at-a-glance"></a>

⚡ At a Glance: Before vs After

Autonomous coding agents burn over 60% of their context window budget on repetitive, low-entropy noise. trimtok compresses this in-process without losing a single line of actionable truth:

| Target Workload | Raw Input | trimtok Compact | Token Reduction | Core Mechanism | | :--- | :---: | :---: | :---: | :--- | | pytest Test Suite (200 tests) | 56,000 chars | 320 chars | -99.4% | Cross-turn stateful delta ([tt:delta]) | | Source Code File (300 functions) | 13,500 toks | 2,400 toks | -82.2% | Multi-language AST skeletonizer | | FastAPI / Python Traceback | 12,400 chars | 890 chars | -92.8% | Crash-site anchor & framework frame prune | | Web Docs & Search Scraping | 15,200 chars | 1,800 chars | -88.1% | Tiered anchor & link de-bloating | | MCP Tool Schemas (25 tools) | 5,800 toks | 780 toks | -86.5% | Bounded JSON-RPC stdio proxy | | Dense Uniform JSON Array | 8,900 chars | 3,100 chars | -65.1% | Lossless columnar TOON v2 encoding | | Mixed-Schema JSON Data | 11,240 chars | 6,630 chars | -41.0% | Multi-schema merged TOON tables | | Build & Compiler Logs | 15,960 toks | 4,869 toks | -69.5% | Adaptive error-rail balanced windowing | | Whole Repo Codebase Map | 12,000 toks | 480 toks | -96.0% | 2k-budget hierarchical tree map |

The 32-payload corpus evaluation is a reproducible fixture run (1,135,441 characters): 76.07% character reduction (863,784 chars) with zero gate failures. Every removed character is verifiable through trimtok_retrieve; eval/audit_honesty.py reports 76.07% claimed and proven, with 0 unproven characters. This corpus result is not a production-traffic estimate.

Production telemetry reports characters, not measured model tokens. usage.jsonl records transform attempts across gates, while audit.jsonl records successful compression events; they are different populations and are rotated independently. Do not combine their totals into one live-traffic percentage or convert characters to tokens without a model-specific tokenizer. Use trimtok gain (optionally with a time window) for the retained usage-log view; older retained logs cannot reproduce historical all-time claims.


Rows in the table are single-payload spot checks over the shapes stored in eval/corpus/; they are not the headline figure. The audited number is the corpus line above, which eval/audit_honesty.py recomputes from the recovery store.

<a id="how-it-works"></a>

🏗️ How It Works

trimtok operates 100% locally and in-process using the Python standard library, RapidFuzz C++ alignment, optional tree-sitter AST parsing (pip install "trimtok[multilang]"), and Zstandard compression. No external daemons, no Docker containers, and zero open network ports. The default install parses Python with the stdlib ast; the other grammar packs are opt-in.

  ┌────────────────────────────────────────────────────────┐
  │ Tier 0: Pre-Flight Context Shield (Zero Disk Footprint)│
  │ ├── Bounded MCP Proxy: filters tools/list schemas      │
  │ └── Codebase Symbol Map: 2k-budget symbol directory map│
  └──────────────────────────┬─────────────────────────────┘
                             ▼
Tool / Terminal Output
         │
         ▼
  ┌────────────────────────────────────────────────────────┐
  │ Tier 1: In-Process Hot-Path (sub-ms typ., guard 200ms) │
  │ ├── 0. Content Policy Gate (secrets / edits verbatim)  │
  │ ├── 1. Cross-Turn Delta Engine (pytest, cargo test)    │
  │ ├── 2. Command-Specific Filters (git, npm, pip, etc.)  │
  │ ├── 3. Structured Traceback Pruning (locks crash-site) │
  │ ├── 4. Lossless & Merged TOON v2 (JSON arrays)         │
  │ ├── 5. Multi-Language AST Skeletonizer (opt-in grammar)│
  │ └── 6. Tiered Web & Doc Link De-bloating               │
  └──────────────────────────┬─────────────────────────────┘
                             │  level: light │ normal │ aggressive
                             │  format: TOON │ minified JSON
            ┌────────────────┴────────────────┐
            ▼                                 ▼
   Agent Context Window             SQLite WAL + Zstandard
   (Clean, dense token stream)     (Tier 2 Recovery via `#c1`)

📑 Table of Contents

| 🏛️ Architecture & Verification | ⚡ Core Engineering Pillars | 🛠️ Setup & Operations | | :--- | :--- | :--- | | • ⚡ At a Glance: Before vs After<br>• 🏗️ How It Works<br>• 🥊 Architecture & Benchmark Matrix<br>• 🔬 Empirical Quality & Fidelity | • Pillar 1: Terminal Compression<br>  ▫ Cross-Turn Delta Engine<br>  ▫ Command-Specific Filters<br>  ▫ Structured Traceback Pruning<br>  ▫ Fault-Signature Folding<br>• Pillar 2: Intake Shield<br>  ▫ Tiered Web & File Intake<br>  ▫ AST Code Skeletonizer<br>  ▫ Lossless TOON v2 (JSON)<br>• Pillar 3: Zero Data Loss<br>  ▫ CCR Recovery (#ref)<br>  ▫ Pointer Tier (pointer_large)<br>  ▫ Telemetry & Diagnostics<br>  ▫ Fail-Open Budget Guards<br>• Pillar 4: Policy & Verification<br>  ▫ Content Policy Gate<br>  ▫ Adaptive Compression Level<br>  ▫ Model-Aware Output Format<br>  ▫ Fix-Accuracy Verification | • First-Time Setup (5 Mins)<br>  ▫ Step 1: Install trimtok<br>  ▫ Step 2: Setup Wizard<br>  ▫ Step 3: Health Verification<br>• Hermes Plugin Setup<br>  ▫ Configuration Options<br>• Slash Commands & CLI<br>  ▫ In-Chat Slash Commands<br>  ▫ KV-Cache Stabilizer (kv-lock)<br>  ▫ Codebase Map (map)<br>  ▫ Bounded MCP Proxy (mcp-proxy)<br>  ▫ Standalone Wrapper (wrap)<br>  ▫ Command Rewriting (rewrite)<br>• Autonomous Agent Ecosystem<br>• Verification & Tests<br>• License (MIT) |


<a id="market-comparison"></a>

🥊 Architecture & Benchmark Matrix: In-Process vs Alternatives

Direct architectural comparison across context compression paradigms:

| Architectural Dimension | trimtok 🌟 | Subprocess CLI (e.g. RTK) | Local HTTP Proxy (e.g. Distil / Headroom) | | :--- | :---: | :---: | :---: | | Execution Layer | In-process hook + CLI | External subprocess fork | Local HTTP daemon / reverse proxy | | Runtime Overhead | sub-ms hook (microsecond in-process call) | 10 – 50 ms (process fork) | proxy queue + network hop | | Data Loss Model | Lossless CCR (SQLite WAL + Zstd) | ❌ Irreversible truncation | ⚠️ Disk cache / ephemeral handles | | Ref Intelligence | ✅ trimtok_exec (compute on #ref) | ❌ None | ⚠️ Whole-block retrieve only | | Cross-Turn State | ✅ [tt:delta] (>95% on reruns) | ❌ None (stateless per command) | ❌ None (stateless stream) | | Structured JSON | ✅ Lossless TOON v2 (55% – 65%) | ⚠️ Lossy head sample (99.6% lossy) | ⚠️ Reversible / dictionary encoder | | Diff Safety Gate | ✅ 100% changed lines preserved | ⚠️ Fixed window line sampling | ⚠️ Token budget truncation | | Prompt Alignment | ✅ kv-lock (KV-Cache friendly) | ❌ None | ⚠️ Sliding window prefix cache | | Network Footprint | ✅ 0 ports (100% in-process) | ✅ 0 ports | ❌ Opens local TCP port |

🎯 Why In-Process Context Intelligence Wins

  • Deterministic & Sub-2ms: Runs directly inside agent runtime hooks with zero daemon processes, zero network ports, and fail-open deadline guards (guard_ms: 200.0).
  • Zero Data Loss Guarantee: Every omitted byte is backed by SQLite WAL + Zstandard. Agents recall slices via trimtok_retrieve or execute Python aggregate queries via trimtok_exec without inflating prompt context.
  • Cross-Turn Stateful Delta: Repeated test suites (pytest, cargo test) and git operations track cross-turn state, surfacing only new regressions and active diffs.

<a id="empirical-verification"></a>

🔬 Empirical Quality & Task Fidelity Verification

Context compression is only valuable if downstream language models retain 100% of reasoning accuracy and task comprehension. To prove that trimtok preserves actionable intelligence without inducing information loss or "lost-in-the-middle" degradation, the pipeline is evaluated across model inference, needle retention, and adversarial stress suites.

1. End-to-End LLM Task Fidelity Benchmark

Evaluated across diverse tool-use tasks (Python tracebacks, function return annotations, lockfile configs, structured logs) by presenting paired identical queries against Raw vs. trimtok-Compressed payloads to coding models:

| Metric | Raw Baseline | trimtok Compressed | Impact / Advantage | | :--- | :---: | :---: | :---: | | Total Prompt Tokens | 22,092 tokens | 10,326 tokens | -53.3% tokens saved | | Task Accuracy | 80.0% (4/5) | 100.0% (5/5) | +20.0% (n=5, indicative only) | | Information Fidelity | 100.0% | 100.0% | Zero hallucinations / zero loss |

Why Compressed Input Yielded Higher Accuracy: On oversized raw payloads, models frequently suffer from Lost-in-the-Middle degradation (e.g. failing to locate function signatures buried inside 4,000+ token raw code files). trimtok's selective skeletonization eliminates noise, enabling the model to achieve higher answer precision on compressed inputs than on uncompressed raw text.

This run is n=5 on hand-built tasks. It shows the mechanism is plausible and that compression did not cost accuracy here; it is not a statistically established accuracy gain. Read the 4/5 → 5/5 delta as indicative, not as a headline number.

2. Needle-In-A-Haystack (NIAH) Retention Benchmark

Concealed critical operational signals (security overrides, replica split-brain fault codes, diagnostic signatures) at depths of 25%, 50% (exact center), and 75% within dense 1,200-line server log streams:

| Needle Depth | Target Payload | Raw Retrieval | Compressed Retrieval | Retention Status | | :---: | :--- | :---: | :---: | :---: | | 25% Depth | `CRITICA

Truncated for display — read the full file on GitHub.

Related Skills

View on GitHub
GitHub Stars3
CategoryAI
Updated1d ago
Forks0

Languages

Python

Trust signals

92/100

From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.

1 low