ablation-planner
Use when main results pass result-to-claim (claim_supported=yes or partial) and ablation studies are needed for paper submission.
Install / Use
npx skills add wanshuiyin/Auto-claude-code-research-in-sleep --skill ablation-plannerInstalls into whichever agent you are using.
SKILL.md
Installable skill definition
Quality Score
Category
MarketingSupported Platforms
Our assessment of ablation-planner
ablation-planner scores 92/100 on our quality scale, 168th of 642 Marketing skills we index (top 27%).
Its SKILL.md is 5.0 KB long, well organised into 18 sections with 2 code examples: a solid amount of guidance for an agent.
With 19,197 GitHub stars, it is one of the more widely adopted skills in the catalogue.
Maintenance, license and trust
- The repository was last updated 2 days ago, so ablation-planner is actively maintained.
- Our last check on 2026-10-27 found the source still online.
- It is released under the MIT license, a permissive license that allows use, modification and commercial use with attribution.
- Its trust signals score 100/100, with no cautions. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.
Safety scan
No issues foundOur scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful.
AI review by kimi-k2.7-code on 2026-09-28. Automated pattern scan on 2026-09-28. It catches known dangerous patterns, not every risk — read a skill before letting an agent act on it.
ablation-planner compared with similar skills
All 4 of these similar skills score higher than ablation-planner; compare them before choosing.
| Skill | Score | Stars | Updated | Format |
|---|---|---|---|---|
| ablation-planner (this skill)by wanshuiyin | 92 | 19.2k | 2d ago | SKILL.md |
| taste-skillby Leonxlnx | 100 | 116.8k | 8d ago | CLAUDE.md |
| Agent-Reachby Panniantong | 100 | 112.3k | 8d ago | CLAUDE.md |
| rufloby ruvnet | 100 | 81.9k | today | CLAUDE.md |
| headroomby headroomlabs-ai | 100 | 79.8k | today | CLAUDE.md |
Frequently asked questions
- How do I install ablation-planner?
- Run
npx skills add wanshuiyin/Auto-claude-code-research-in-sleep --skill ablation-planner. The install tabs above show the steps for each supported agent. - Which AI agents does ablation-planner work with?
- It is written for OpenAI Codex, as a SKILL.md file. Other agents that read the same format can often use it too.
- Is ablation-planner safe to use?
- Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful. It is MIT-licensed and scores 100/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
- Is ablation-planner still maintained?
- The repository was last updated 2 days ago, so ablation-planner is actively maintained.
Skill content
View source on GitHubname: ablation-planner description: "Use when main results pass result-to-claim (claim_supported=yes or partial) and ablation studies are needed for paper submission." argument-hint: "[method-description-or-claim]" allowed-tools: Bash(*), Read, Grep, Glob, Write, Edit, mcp__codex__codex, mcp__codex__codex-reply
Ablation Planner
Systematically design ablation studies that answer the questions reviewers will ask. Codex leads the design (reviewer perspective), CC reviews feasibility and implements.
Context: $ARGUMENTS
When to Use
- Main results pass
/result-to-claimwith claim_supported = yes or partial - User explicitly requests ablation planning
/auto-review-loopreviewer identifies missing ablations
Workflow
Step 1: Prepare Context
CC reads available project files to build the full picture:
- Method description and components (from
idea-stage/docs/research_contract.md, legacydocs/research_contract.md, or project CLAUDE.md) - Current experiment results (from EXPERIMENT_LOG.md, EXPERIMENT_TRACKER.md, or W&B)
- Confirmed and intended claims (from result-to-claim output or project notes)
- Available compute resources (from CLAUDE.md server config, if present)
Step 2: Codex Designs Ablations
mcp__codex__codex:
model: gpt-6-astra
config: {"model_reasoning_effort": "xhigh"}
prompt: |
You are a rigorous ML reviewer planning ablation studies.
Given this method and results, design ablations that:
1. Isolate the contribution of each novel component
2. Answer questions reviewers will definitely ask
3. Test sensitivity to key hyperparameters
4. Compare against natural alternative design choices
Method: [description from project files]
Components: [list of removable/replaceable components]
Current results: [key metrics from experiments]
Claims: [what we claim and current evidence]
For each ablation, specify:
- name: what to change (e.g., "remove module X", "replace Y with Z")
- what_it_tests: the specific question this answers
- expected_if_component_matters: what we predict if the component is important
- priority: 1 (must-run) to 5 (nice-to-have)
Also provide:
- coverage_assessment: what reviewer questions these ablations answer
- unnecessary_ablations: experiments that seem useful but won't add insight
- suggested_order: run order optimized for maximum early information
- estimated_compute: total GPU-hours estimate
Step 3: Parse Ablation Plan
Normalize Codex response into structured format:
## Ablation Plan
### Component Ablations (highest priority)
| # | Name | What It Tests | Expected If Matters | Priority |
|---|------|---------------|---------------------|----------|
| 1 | remove module X | contribution of X | performance drops on metric Y | 1 |
| 2 | replace X with simpler Z | value of learned vs fixed | drops, especially on dataset A | 2 |
### Hyperparameter Sensitivity
| # | Parameter | Values to Test | What It Tests | Priority |
|---|-----------|---------------|---------------|----------|
| 3 | lambda | [0.01, 0.1, 1.0] | sensitivity to regularization | 3 |
### Design Choice Comparisons
| # | Name | What It Tests | Priority |
|---|------|---------------|----------|
| 4 | joint vs separate matching | whether joint adds value | 4 |
### Coverage Assessment
[What reviewer questions these ablations answer]
### Unnecessary Ablations
[Experiments that seem useful but won't add insight — skip these]
### Run Order
[Optimized for maximum early information]
### Estimated Compute
[Total GPU-hours]
Step 4: CC Reviews Feasibility
Before running anything, CC checks:
- Compute budget: can we afford all ablations with available GPUs?
- Code changes: which ablations need code modifications vs config-only changes?
- Dependencies: which ablations can run in parallel?
- Cuts: if budget is tight, propose removing lower-priority ablations and ask Codex to confirm
Step 5: Implement and Run
- Create configs/scripts for each ablation (config-only changes first)
- Smoke test each ablation before full run
- Run in suggested order, using descriptive names (e.g.,
ablation-no-module-X) - Track results in EXPERIMENT_LOG.md
- After all ablations complete → update findings.md with insights
Rules
- Codex leads the design. CC does not pre-filter or bias the ablation list before Codex sees it. Codex thinks like a reviewer; CC thinks like an engineer.
- Every ablation must have a clear
what_it_testsandexpected_if_component_matters. No "just try it" experiments. - Config-only ablations take priority over those needing code changes (faster, less error-prone).
- If total compute exceeds budget, CC proposes cuts and asks Codex to re-prioritize — don't silently drop ablations.
- Component ablations (remove/replace) take priority over hyperparameter sweeps.
- Do not generate ablations for components identical to the baseline (no-op ablations).
- Record all ablation results in EXPERIMENT_LOG.md, including negative results (component removal had no effect = important finding).
Related Skills
taste-skill
116.8kAgent skills that give coding agents taste: art direction, reference images and a browser check instead of generic AI pages
Agent-Reach
112.3kGive your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.
ruflo
81.9k🌊 The original agent harness. Deploy intelligent multi-player swarms, coordinate autonomous workflows, and build conversational AI systems. Features adaptive memory, self-learning intelligence, federation, vector RAG integration, and native Claude Code / Codex / Hermes and many more Integrated
headroom
79.8kCompress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
Languages
Trust signals
From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.
