crit
Agentic toolkit for building and optimizing high-performance inference engines tailored to your GPU and desired LLM.
Install / Use
claude mcp add crit-labs -- npx -y github:crit-labs/critIf the server publishes to npm under a different name, use that package instead — check the repo README.
MCP Server
Model Context Protocol server
Quality Score
Category
AI & Machine LearningSupported Platforms
Tags
Our assessment of crit
crit scores 82/100 on our quality scale, 694th of 947 AI & Machine Learning skills we index.
Its MCP Server is 34 KB long, well organised into 8 sections with 3 code examples: a thorough specification that gives an agent plenty to work with.
It has 48 GitHub stars, so there is little community track record yet; judge it on its content.
Maintenance, license and trust
- The repository was last updated today, so crit is actively maintained.
- Our last check on 2026-11-13 found the source still online.
- It is released under the Apache-2.0 license, a permissive license that allows use, modification and commercial use with attribution.
- Its trust signals score 97/100, with no cautions. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.
Safety scan
No issues foundOur scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful.
AI review by kimi-k2.7-code on 2026-11-10. Automated pattern scan on 2026-11-10. It catches known dangerous patterns, not every risk — read a skill before letting an agent act on it.
crit compared with similar skills
All 4 of these similar skills score higher than crit; compare them before choosing.
| Skill | Score | Stars | Updated | Format |
|---|---|---|---|---|
| crit (this skill)by crit-labs | 82 | 48 | today | MCP Server |
| claude-memby thedotmack | 100 | 117.6k | today | CLAUDE.md |
| Agent-Reachby Panniantong | 100 | 112.3k | 8d ago | CLAUDE.md |
| Understand-Anythingby Egonex-AI | 100 | 92.8k | 1d ago | CLAUDE.md |
| rufloby ruvnet | 100 | 81.9k | today | CLAUDE.md |
Frequently asked questions
- How do I install crit?
- Run
claude mcp add crit-labs -- npx -y github:crit-labs/crit. The install tabs above show the steps for each supported agent. - Which AI agents does crit work with?
- It is written for Claude Code, Claude Desktop and OpenAI Codex, as a MCP Server file. Other agents that read the same format can often use it too.
- Is crit safe to use?
- Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful. It is Apache-2.0-licensed and scores 97/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
- Is crit still maintained?
- The repository was last updated today, so crit is actively maintained.
Skill content
View source on GitHubWhy wait for someone to optimize an LLM for your GPU when an AI agent can build a specialized inference engine in a few hours?
Summary • Quick start • What crit solves • Case studies •
How crit works • API • CLI • crit-hub • Limitations • Documentation
<a id="summary"></a>
Summary
crit is a toolkit for optimizing or building inference engines tailored to your hardware and desired model.
[!IMPORTANT] Combines a runtime foundation, reusable techniques as plugins, kernel discovery and physics-based evaluation.
In our latest case study, a coding agent built an engine for Qwen3.8-27B on one RTX 5090 in under 4 hours and five candidates, reaching 189 tok/s sampled decode and 8,295 tok/s prefill with the model's full 262,144-token context.
Follow the building guide yourself, or give your coding agent the agent instructions, target model and hardware.
<a id="quick-start"></a>
Quick start
Give one of these prompts to your coding agent and specify your target LLM or target engine:
Use github.com/crit-labs/crit to build an optimized inference engine for <huggingface_url>.
Use github.com/crit-labs/crit to optimize my own inference engine in <engine_reference>.
<a id="what-crit-solves"></a>
What crit solves
The boring stuff has already been built ⏱️
Requests, memory management and cleanup eat up development time before optimization even starts.
critsupplies a clean foundation so you can get to the work that makes your engine fast.
Curated high-impact inference strategies 📊
Don't rebuild inference strategies from scratch. We studied established inference engines and distilled their core speedup strategies into reusable code, so you can apply what works to your model and hardware quickly. Each strategy is a plugin with its own tests, and an ablation switches each one off to measure what it adds to your engine.
Physics-based evaluation suite 🔬
A benchmark score alone doesn't tell you what's still left on the table.
critmeasures your hardware's limits and shows how close your engine gets, with quality and correctness checks along the way.
<a id="case-study"></a>
Case Studies
[!NOTE] The engines of these case studies were built with crit 0.16, before the plugin architecture. They are being rebuilt as plugin packages and return to
crit-hubas they are.
We gave Claude Opus 5.5 a target LLM along with the crit repository and let it cook autonomously.
| What it shows | Evidence |
| --- | --- |
| Fast engines | Qwen3.8-27B on one RTX 5090: 189 tok/s sampled decode, 4.4x its own first working candidate, and 8,295 tok/s prefill, with the full 262,144-token context |
| Quality is your call | The same model built once per quality mode: speed prefills 2.3x faster, preserve stays within 0.4% of the exact reference's perplexity, and both decode at the same speed |
| Hours, not weeks | 3h 55m and five candidates from the run's first command to the selected engine, $51 at API list prices |
| Every step is evidence | Each candidate is sealed with its exact source and its Eval result, so the tables below are a record, not a recollection |
| Honest about what is left | Decode runs at 53% of the ceiling Eval measures for it, prefill at 46%; Eval reports each gap next to the score |
| Next | Gemma 4 31B (dense), and Qwen3.8-Flash-Next and Nemotron 3.5 Super 120B, mixture-of-experts models larger than VRAM |
| Part | Spec | | --- | --- | | GPU | NVIDIA GeForce RTX 5090, 32 GB, 450 W power limit | | PCIe | Gen 4 x16 (the card supports Gen 5; the board's slot is Gen 4) | | CPU | AMD Ryzen 5 9600, 6 cores, 12 threads | | RAM | 96 GB DDR5 (2x 48 GB) at 4,800 MT/s | | OS | Windows 11 Home (build 26200), engines run in WSL 2 on Ubuntu 24.04 | | Driver and CUDA | NVIDIA driver 596.49 (CUDA 13.2), CUDA toolkit 13.1 |
</details><details> <summary><strong>📋 Dense LLM, Qwen architecture</strong></summary> <br />
Goal: Get to a fast, working engine for a dense model as quickly as possible. Limit: A fixed candidate budget.
<div align="center"> <table> <tr> <td align="center" width="16%"> <strong>3h 55m</strong>
Development time </td> <td align="center" width="16%"> <strong>
189.0 tok/s</strong>
Sampled decode </td> <td align="center" width="16%"> <strong>
8,295 tok/s</strong>
Prefill </td> <td align="center" width="16%"> <strong>
154 ms</strong>
TTFT </td> <td align="center" width="16%"> <strong>
+3.6%</strong>
Perplexity vs reference </td> <td align="center" width="16%"> <strong>
0.134</strong>
Mean KL divergence </td>
</tr> </table> </div> <br />mtp4-nvfp4-mma
runs Qwen3.8-27B in its original NVFP4 weights on one RTX 5090, with the model's full 262,144
tokens of context, in the speed quality mode: the engine may trade more
drift from the checkpoint for speed (crit-eval-v0.2.1, which measured it, judged that mode against the
checkpoint run with the 4-bit and 8-bit activations it declares). One coding agent built it on crit's current main as a Python engine on Core:
cuBLASLt for the prompt's 4-bit and 8-bit matrix multiplies, flash-linear-attention for the
gated-delta layers, and its own Triton and CUDA kernels for attention and decode. Each round drafts 4
tokens with the model's own built-in draft layer and verifies them with rejection sampling on the
GPU, so sampled requests draft too. Its own tensor-core NVFP4 GEMV reads the 4-bit weights at
1.2–1.4 TB/s, against 0.9–1.0 TB/s for cuBLASLt at the same shapes.
| Candidate | What changed | Sampled decode | Prefill | TTFT | Dev time, cumulative | Agent tokens, cumulative | API equivalent, cumulative | | --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | | 1 | Working engine: reads the checkpoint as stored, 8-bit KV cache, one CUDA graph per token, no drafting | 43 tok/s | 6,027 tok/s | 290 ms | 0h 55m | 39.2M | $13.96 | | 2 | Drafts 3 tokens ahead with the model's draft layer, 8-bit decode GEMVs, fused prompt kernels (missed a gate: short prompts reached their first token 12% later at 65,536 tokens of context than at 262,144) | 134 tok/s | 7,771 tok/s | 219 ms | 1h 49m | 69.6M | $23.26 | | 3 | Drafts 4 tokens ahead; short prompts run as captured CUDA graphs | 155 tok/s | 7,660 tok/s | 169 ms | 2h 44m | 97.7M | $31.51 | | 4 | Its own tensor-core NVFP4 GEMV for decode and verify, using the GPU's FP4 conversion instruction | 181 tok/s | 8,218 tok/s | 156 ms | 3h 27m | 135.3M | $47.15 | | 5 | Drafts from NVFP4 copies of the draft layer and output head; the verify pass keeps the stored weights | 189 tok/s | 8,295 tok/s | 154 ms | 3h 55m | 149.3M | $50.75 |
The last three columns are running totals from the run's first command (06:28 UTC) to each
candidate's seal, and for candidate 5 to its selection, so its row is the whole run. The run spent
its budget of three candidates on 1 to 3, then resumed for two more; development time leaves out the
3h 26m in between. Candidates 1 to 3 were built from crit and crit-eval alone; before planning 4 and
5 the agent had read the notes of earlier runs on this model. Agent tokens and API equivalent count
the agent's messages in that span, from its Claude Code session transcript: Claude Opus 5.5 for
every message, priced at Anthropic's API list prices ($4/MTok input, $20/MTok output, $8/MTok 1-hour
cache writes, $0.20/MTok cache reads; 99% of the tokens are cache reads).
Measured with crit-eval (crit-eval-v0.2.1) at the full 262,144-token context. Decode is sampled at the
model's own settings (temperature 1.0, top-p 0.95, top-k 20), the geometric mean over six prompts,
from a chat question to a 14K-token agent session; Eval decodes each engine's own output.
- Long context: a prompt that fills all 262,144 tokens gets its first token after 144 s, then decodes at 110 tok/s. On prompts outside Eval's speed suite, in 8 languages and code, it decodes at 131 tok/s.
- Quality: the worst path uses 70% of the
speeddrift budget. Greedy decoding reaches 194 tok/s, and drafting never changes the output: it is token for token what plain decoding produces. - What is left: a verify pass takes 15.1 ms against the 12.5 ms its weight reads need, and a draft step 1.24 ms against 0.62 ms. Eval projects 251 tok/s if both reached their floors.
<details> <summary><strong>📋 Quality modes comparison</strong></summary> <br />
Goal: Show what each quality mode buys, on the same model, GPU and agent. Limit: Five candidates per run.
crit judges an engine's quality in one of two modes, chosen per run: how much output drift from the
checkpoint you select the engine may trade for speed, whatever format that checkpoint is stored in.
speed, the default: the engine picks its precision (4-bit and 8-bit
activations, a lower-precision KV cache or head) within a budget that only keeps out a broken engine
(mean KLD 0.4), and output differences that part at near-ties are allowed. preserve holds the output
close to the checkpoint's, within a 5% drift budget, so the prompt's activations stay in 16 bits unless
quality shows they can go lower. (The two engines below were built when the speed budget was 10% and
that mode judged against the checkpoint run with its declared activations; today both modes judge
against the same exact reference.) We built Qwen3.8-27B on one RTX
5090 once in each mode: mtp4-nvfp4-mma
from the case study above, and
mtp-adaptive-width,
which moves between 2 and 8 drafts per round: wider after a round that kept every draft, narrower
after one that kept fewer.
| | speed | preserve |
| --- | ---: | ---: |
| Sampled decode | **189.0 tok/s
Truncated for display — read the full file on GitHub.
Related Skills
claude-mem
117.6kPersistent Context Across Sessions for Every Agent – Captures everything your agent does during sessions, compresses it with AI, and injects relevant context back into future sessions. Works with Claude Code, OpenClaw, Codex, Gemini, Hermes, Copilot, OpenCode + More
Agent-Reach
112.3kGive your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.
Understand-Anything
92.8kGraphs that teach > graphs that impress. Turn any code into an interactive knowledge graph you can explore, search, and ask questions about. Works with Claude Code, Codex, Cursor, Copilot, Gemini CLI, and more.
ruflo
81.9k🌊 The original agent harness. Deploy intelligent multi-player swarms, coordinate autonomous workflows, and build conversational AI systems. Features adaptive memory, self-learning intelligence, federation, vector RAG integration, and native Claude Code / Codex / Hermes and many more Integrated
Languages
Trust signals
From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.
