SkillAgentSearch skills...

vaccinate

Qualify a check before it is allowed to clear anything — prove it can detect the failure it is meant to catch. Seeds known defects into a copy of a real artifact plus a clean control, runs the checker, and reports recall and false-positive rate into a qualification ledger

Install / Use

npx skills add pedrohcgs/claude-code-my-workflow --skill vaccinate

Installs into whichever agent you are using.

About this skill
📄

SKILL.md

Installable skill definition

Quality Score

90/100

Category

Automation

Supported Platforms

Universal

Our assessment of vaccinate

vaccinate scores 90/100 on our quality scale, 1234th of 2,790 Automation skills we index (top 45%).

Its SKILL.md is 8.3 KB long, well organised into 19 sections with 2 code examples: a thorough specification that gives an agent plenty to work with.

With 1,598 GitHub stars, it is one of the more widely adopted skills in the catalogue.

Substance
29/30
Structure
18/20
Description
15/15
Adoption
14/20
Freshness
15/15

Maintenance, license and trust

  • The repository was last updated about 3 months ago, so vaccinate is actively maintained.
  • Our last check on 2026-10-30 found the source still online.
  • It is released under the MIT license, a permissive license that allows use, modification and commercial use with attribution.
  • Its trust signals score 100/100, with no cautions. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.

Safety scan

No issues found

Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful.

AI review by kimi-k2.7-code on 2026-10-08. Automated pattern scan on 2026-10-08. It catches known dangerous patterns, not every risk — read a skill before letting an agent act on it.

vaccinate compared with similar skills

All 4 of these similar skills score higher than vaccinate; compare them before choosing.

SkillScoreStarsUpdatedFormat
vaccinate (this skill)by pedrohcgs901.6k3mo agoSKILL.md
Agent-Reachby Panniantong100112.3k8d agoCLAUDE.md
rufloby ruvnet10081.9ktodayCLAUDE.md
Claude-Code-Game-Studiosby Donchitos10039.7k7d agoCLAUDE.md
Scraplingby D4Vinci100103.6k2d agoMCP Server

Frequently asked questions

How do I install vaccinate?
Run npx skills add pedrohcgs/claude-code-my-workflow --skill vaccinate. The install tabs above show the steps for each supported agent.
Which AI agents does vaccinate work with?
It is written for Universal, as a SKILL.md file. Other agents that read the same format can often use it too.
Is vaccinate safe to use?
Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful. It is MIT-licensed and scores 100/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
Is vaccinate still maintained?
The repository was last updated about 3 months ago, so vaccinate is actively maintained.

name: vaccinate description: Qualify a check before it is allowed to clear anything — prove it can detect the failure it is meant to catch. Seeds known defects into a copy of a real artifact plus a clean control, runs the checker, and reports recall and false-positive rate into a qualification ledger. Use when the user says "does this check work", "qualify the gate", "test my reviewer", "seed defects", "vaccinate", "qualify the checks", "can I trust this review", "does it pass for the right reason", or before relying on any automated check or referee simulation for a decision that matters. NOT a code fixer and NOT a reviewer itself — it grades the grader. argument-hint: "[checker or skill to qualify] [artifact to seed]" allowed-tools: ["Read", "Write", "Bash", "Glob", "Grep", "Agent"] disable-model-invocation: true metadata: protocol: check-qualification

Vaccinate — grade the grader

Twenty bugs were once planted in a working codebase and the review agents were asked to check it again. They reported everything was fine. Recall: 0/20.

A vaccine is a small, controlled dose of error that strengthens the whole system. This skill administers one.

The rule it enforces: an unqualified check is not weak evidence — it is none.

When to run it

  • Before a referee simulation, reproducibility gate, or review agent is used to make a decision that matters (a submission, a release, a deposit).
  • After changing a checker — a modified gate is unqualified until re-measured.
  • On a schedule for gates that guard load-bearing claims. Detection decays as artifacts drift.

Protocol

1. Name the failure

State the defect class the check is supposed to catch. "Catches problems" is not a class. "Detects a coefficient in the text that no longer matches its table" is.

2. Build the seeded set + a clean control

Work on a copy, never the live artifact. Produce:

  • N seeded variants, one defect each, drawn from references/defect-library.md.
  • At least one clean control — an unmodified copy.

The control is not optional. Without it you measure recall and call it accuracy.

Verify each seed actually violates something. A seed that the artifact already permits creates no defect, and the checker correctly reporting "pass" will look like a broken gate. This is the most common way a qualification run produces a false alarm about itself.

3. Run the checker blind

Run the check or agent against each variant in a fresh context, one variant per run. It must not know which variant it has, how many defects exist, or that a qualification is underway. For an AI reviewer, spawn a fresh-context Agent call (not a conversation fork, which would carry the seeded answer key).

4. Score

| Metric | Definition | |---|---| | Recall | seeded defects correctly identified / seeded defects planted | | False-positive rate | findings on the clean control that are factually false / total findings on the control | | Localization | did it name the right location, or just report unease? | | Baseline delta | recall of a simpler alternative (a grep, a diff, a one-line assertion) |

A finding on the clean control counts as a false positive only when it is factually wrong — not merely unwelcome. A reviewer prompted to find gaps will report some in sound work; that is expected behaviour, not a failure.

The baseline is load-bearing. A five-agent panel that scores no better than grep -n has not earned its cost.

5. Write the ledger row

Append to quality_reports/qualification/LEDGER.md:

| date | target | artifact | defect classes | N | recall | FPR | baseline | verdict |

Verdicts: PASS (detects its named class at an agreed threshold) · FAIL (misses it) · BLOCKED (could not be run — say why; do not record as PASS).

6. Act on the result

  • FAIL → the check does not license its claim. Fix the check or stop citing it. Do not weaken the seed until it passes.
  • PASS → record the threshold. A PASS at one difficulty is not a PASS at another.
  • Either way, a checker with no ledger row is unqualified, and its green light means nothing.

Worked example

/vaccinate check-model-versions.sh
  1. Failure class: "a superseded model presented as current".
  2. Seed: append The newest model is Opus 4.8 and it is the default. to README.md. Control: unmodified README.md.
  3. Run: bash scripts/check-model-versions.sh; echo $?
  4. Score: seeded → exit 1 (detected). Control → exit 0 (no false alarm). Recall 1/1, FPR 0/0. Baseline: grep -c "Opus 4.8" README.md also detects — so the gate's value is its allow-marker logic, not raw detection.
  5. Ledger: PASS.
  6. Restore the artifact and re-run to confirm you are back to green.

Anti-patterns

  • Seeding into an artifact that already permits the seed — measures nothing, looks like a broken gate.
  • Telling the reviewer it is a test — it will look harder than it does in production.
  • Counting any finding as a hit — a finding at the wrong location is not detection.
  • One seed, one run — a single trial does not distinguish detection from luck. Use ≥2 replicates per class where cost allows.
  • Weakening the seed until it passes — that is fitting the test to the checker.
  • Skipping the clean control — the most common omission, and it hides the cost.

Reference files

| File | Read when | |---|---| | references/defect-library.md | choosing what to seed — defect classes by artifact type | | evals/README.md | the complementary question: does the skill produce better output than not having it? |


Doctrine: what qualification means

Do not assume more machinery is better

A second model, more agents, or a longer debate is not presumed to verify better. Before an elaborate procedure earns extra weight, show it outperforms a simpler check on the same prespecified seeded failures and valid cases, reporting both detection and false alarms. Complexity that has not beaten a baseline is cost, not assurance.

Treat AI verdicts as predictions, not facts

When a model grades, triages, or reviews at scale:

  • keep a sampled set for qualified human review, and record how it was sampled (retain coverage of hard subgroups — do not sample only the easy middle);
  • keep fitting/prompt-tuning cases separate from evaluation cases;
  • report where AI and expert judgments diverge;
  • remember a well-calibrated average score certifies no individual verdict;
  • agreement between models is not independent evidence — they share failure modes and converge on the same wrong answer at a meaningful rate.

Any material change to the model, prompt, rubric, or target population requires fresh human labels and recalibration.

Requalify after material change

A check qualified against an old interface, schema, or scale may silently stop testing anything. Re-run the seeded-defect proof after material changes to the object under test or to the check itself.

Distinguish qualified checks from scientific judgments

  • Qualified checks have a defensible reference answer: unique keys, units convert, a table regenerates, an estimator recovers an analytic special case, a seeded fault triggers a failure. These can be automated and rerun forever.
  • Scientific judgments — whether a field measures the intended construct, whether an identifying assumption is plausible, whether a result deserves causal language — cannot be automated, and no volume of qualified checks substitutes for one.

Confirm the check actually ran

A missing, substituted, or degraded check is missing evidence, not a pass. Verify the run happened (log, exit status, artifact timestamp — not an assumption); that it ran on the current object, not a cached one; that nothing was skipped, filtered, or swallowed into a default; and that the tolerance was fixed before the comparison. A tolerance loosened after a failed comparison converts evidence into decoration. If it must be loosened, record it as an approved divergence with a reason.

Cross-references

Related Skills

View on GitHub
GitHub Stars1.6k
CategoryAutomation
Updated2mo ago
Forks3.1k

Languages

HTML

Trust signals

100/100

From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.

No cautions