challenge
Stress-test a finding against the choices you did not make. Enumerates the discrete forks a competent analyst could have taken (measure definition, sample filter, control set, clustering level, weighting, functional form), runs the specification grid, and reports the distribution rather than a pointβ¦
Install / Use
npx skills add pedrohcgs/claude-code-my-workflow --skill challengeInstalls into whichever agent you are using.
SKILL.md
Installable skill definition
Quality Score
Category
Development & EngineeringSupported Platforms
Tags
Our assessment of challenge
challenge scores 85/100 on our quality scale, 2627th of 4,700 Development & Engineering skills we index.
Its SKILL.md is 7.5 KB long, well organised into 11 sections and no code examples: a thorough specification that gives an agent plenty to work with.
With 1,598 GitHub stars, it is one of the more widely adopted skills in the catalogue.
Maintenance, license and trust
- The repository was last updated about 3 months ago, so challenge is actively maintained.
- Our last check on 2026-11-04 found the source still online.
- It is released under the MIT license, a permissive license that allows use, modification and commercial use with attribution.
- Its trust signals score 100/100, with no cautions. These come from repository metadata, not a code audit β read the skill file before letting an agent act on it.
Safety scan
No issues foundOur scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful.
AI review by kimi-k2.7-code on 2026-10-07. Automated pattern scan on 2026-10-07. It catches known dangerous patterns, not every risk β read a skill before letting an agent act on it.
challenge compared with similar skills
All 4 of these similar skills score higher than challenge; compare them before choosing.
| Skill | Score | Stars | Updated | Format |
|---|---|---|---|---|
| challenge (this skill)by pedrohcgs | 85 | 1.6k | 3mo ago | SKILL.md |
| taste-skillby Leonxlnx | 100 | 116.8k | 8d ago | CLAUDE.md |
| ai-job-searchby ai-job-search-dev | 100 | 72.1k | today | CLAUDE.md |
| claude-howtoby luongnv89 | 100 | 47.3k | 2d ago | CLAUDE.md |
| lobehubby lobehub | 100 | 85.6k | today | MCP Server |
Frequently asked questions
- How do I install challenge?
- Run
npx skills add pedrohcgs/claude-code-my-workflow --skill challenge. The install tabs above show the steps for each supported agent. - Which AI agents does challenge work with?
- It is written for Universal, as a SKILL.md file. Other agents that read the same format can often use it too.
- Is challenge safe to use?
- Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful. It is MIT-licensed and scores 100/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
- Is challenge still maintained?
- The repository was last updated about 3 months ago, so challenge is actively maintained.
Skill content
View source on GitHubname: challenge description: Stress-test a finding against the choices you did not make. Enumerates the discrete forks a competent analyst could have taken (measure definition, sample filter, control set, clustering level, weighting, functional form), runs the specification grid, and reports the distribution rather than a point estimate β then attacks the identifying assumption with named, computable sensitivity statistics. Use when the user says "is this robust", "challenge this result", "specification curve", "multiverse", "how sensitive is this", "what if I'd used a different measure", "stress-test my estimate", or before a result becomes a headline claim. NOT a reviewer of prose or code β it challenges the CLAIM. argument-hint: "[script or results file] [--forks N] [--dry-run]" allowed-tools: ["Read", "Grep", "Glob", "Bash", "Write", "Edit", "Agent", "Task"] disable-model-invocation: true metadata: protocol: threat-prioritization
Challenge β does the result survive the choices you didn't make?
A single specification is one draw from a distribution you never looked at.
Why this exists, measured rather than asserted. In a controlled study, 150 autonomous agents were given the same data and the same questions. Effect-size interquartile ranges reached ~10.7 %/yr, and the spread concentrated in discrete measure-choice forks β not in estimation noise. Within a measure family, agents agreed to ~0.25 %/yr. Two findings from that study shape this skill:
- AI peer review left the spread essentially unchanged. Review catches errors; it does not reduce analytical-choice variance. A clean referee report is not robustness.
- Exposure to exemplar papers collapsed the spread by 80β99 % β convergence by imitation, not by correctness. Herding is not agreement.
So the spread has to be measured, not reviewed away.
Preconditions
- A working baseline specification that runs and produces the headline estimate.
- The estimate's estimand stated in words β "the ATT for units treated in 2015, over 2013β2019, on the treated population". If you cannot state it, stop: you cannot challenge a claim you have not defined.
- A fork budget (
--forks, default 64). Grid size is the product of your choices; it grows faster than intuition.
Step 1 β Enumerate the forks, before running anything
List every point where a competent, honest analyst could have chosen differently. Do this before seeing any alternative result, and write it down β the list is the pre-registration of the challenge.
| Fork | Typical alternatives | |---|---| | Measure definition | level vs rate vs share; dollar vs count; stock vs flow | | Sample filter | balanced vs unbalanced; trimming rules; inclusion windows | | Control set | none / baseline / baseline+trends / interacted | | Clustering level | unit / treatment-assignment / two-way | | Weighting | unweighted / population / inverse-propensity | | Functional form | levels / logs / IHS / Poisson | | Winsorization | none / 1% / 5% |
Some forks change the estimand, not just the estimate β averaging over them is meaningless.
references/fork-catalog.mdlabels every fork; record estimand forks separately and say so in the report.
Ship --dry-run first. Print the grid size and an estimated runtime before executing
anything. A 6-fork grid with 3 options each is 729 fits.
Step 2 β Run the grid
One fit per cell, same seed, same data build. Persist every cell β including failures. A specification that does not converge is information about fragility, not a cell to drop.
Record per cell: the fork coordinates, the point estimate, the standard error, N, and the convergence status.
Step 3 β Report the distribution, not the winner
- Specification curve: estimates sorted, with the fork coordinates shown underneath so a reader can see which choices move the result.
- The share of specifications with the same sign, and the share significant at conventional levels. Report both; they answer different questions.
- Which fork drives the spread. This is the payload. "The result is robust except to the choice between dollar and share volume" is a far more useful sentence than a robustness paragraph.
- Your baseline's percentile in its own distribution. If the headline sits at the 97th percentile of specifications you yourself called defensible, say so.
The descriptive curve is not a test β read it as a description of fragility. If you need inference over the whole curve, use specification-curve analysis's joint permutation test (Simonsohn, Simmons & Nelson 2020), which supplies the sharp null the picture alone lacks.
Step 4 β Attack the identifying assumption
The grid varies what you can vary. The identifying assumption is what you cannot test β so
bound it instead, with a named, computable statistic.
See references/sensitivity-statistics.md.
| Concern | Statistic | |---|---| | Unobserved confounding | E-value; CinelliβHazlett robustness value | | Selection on observables β unobservables | Oster Ξ΄ (with a stated RΒ²max) |
Rows for the causal-identification designs are deliberately absent (unvetted-methods veto): populate them from your field's canonical sources after vetting.
Label every statistic executable-here or describe-and-cite. Honesty about what your
environment can actually run is itself a verification step; a cited-but-unrun statistic is not
evidence.
Step 5 β Placebo and falsification
Where a falsification test exists, run it: a negative-control outcome that should show nothing, a negative-control exposure, a timing placebo. A passed placebo is weak positive evidence; a failed placebo is strong negative evidence. Report both with equal prominence.
Step 6 β Write the ledger entry
Append to the specification-search ledger (verification-ladder.md
rung 5): the fork list, the grid size, the distribution summary, which forks moved the result,
the sensitivity statistics with their values, and every attempt including the failures.
Pre-commit the interpretation. Before running the grid, write down what result would SUPPORT and what would WEAKEN the claim. The ledger is the arbiter. A robustness exercise interpreted after the fact is not a robustness exercise.
Anti-patterns
- Running until something interesting appears. That is the pathology this skill exists to prevent. Fixed fork list, fixed budget, stated stopping rule.
- Reporting only the specifications that agree β a curve showing only supporting cells is a fishing expedition with better graphics.
- Treating a wide curve as failure. Wide is a finding. Publish it and say what drives it.
- Adding forks nobody would defend to pad the denominator and dilute the fragile cells.
Reference files
| File | Read when |
|---|---|
| references/sensitivity-statistics.md | a challenge rests on an untestable identifying assumption and needs a computable bound |
| references/fork-catalog.md | enumerating forks for a design you have not challenged before |
Cross-references
verification-ladder.mdβ rung 4 (analytic verification) and rung 5 (the ledger)/simulation-studyβ when the question is finite-sample performance, not robustness/preregisterβ reserve a holdout before the search, not after
Related Skills
taste-skill
116.8kAgent skills that give coding agents taste: art direction, reference images and a browser check instead of generic AI pages
ai-job-search
72.1kThe job search that runs on your machine. AI job application framework built on Claude Code: evaluate postings, tailor CVs, write cover letters, prep interviews. Fork it and own it.
claude-howto
47.3kA visual, example-driven guide to Claude Code β from basic concepts to advanced agents, with copy-paste templates that bring immediate value.
lobehub
85.6kπ€― LobeHub - Super-Agents on Your Team. Add super-agents to the chats your team already uses.They take on work by themselves and get better every week.
Languages
Trust signals
From repository metadata: license, adoption, age and documentation. Not a code audit β see the Safety scan above for what the skill file itself contains.
