SkillAgentSearch skills...

skills-validate

Use when explicitly asked to run a live or plan-only application test for a published Scenario skill.

Install / Use

npx skills add scenario-labs/skills --skill skills-validate

Installs into whichever agent you are using.

About this skill
📄

SKILL.md

Installable skill definition

Quality Score

88/100

Supported Platforms

Universal

Tags

Our assessment of skills-validate

skills-validate scores 88/100 on our quality scale, 1968th of 4,698 Development & Engineering skills we index (top 42%).

Its SKILL.md is 12 KB long, well organised into 13 sections with 2 code examples: a thorough specification that gives an agent plenty to work with.

With 1,048 GitHub stars, it is one of the more widely adopted skills in the catalogue.

Substance
30/30
Structure
18/20
Description
12/15
Adoption
13/20
Freshness
15/15

Maintenance, license and trust

  • The repository was last updated 29 days ago, so skills-validate is actively maintained.
  • It is released under the MIT license, a permissive license that allows use, modification and commercial use with attribution.
  • Its trust signals score 100/100, with no cautions. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.

Safety scan

No issues found

Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful.

AI review by kimi-k2.7-code on 2026-11-10. Automated pattern scan on 2026-11-10. It catches known dangerous patterns, not every risk — read a skill before letting an agent act on it.

skills-validate compared with similar skills

All 4 of these similar skills score higher than skills-validate; compare them before choosing.

SkillScoreStarsUpdatedFormat
skills-validate (this skill)by scenario-labs881.0k29d agoSKILL.md
taste-skillby Leonxlnx100116.8k8d agoCLAUDE.md
ai-job-searchby ai-job-search-dev10072.1ktodayCLAUDE.md
claude-howtoby luongnv8910047.3k2d agoCLAUDE.md
lobehubby lobehub10085.6ktodayMCP Server

Frequently asked questions

How do I install skills-validate?
Run npx skills add scenario-labs/skills --skill skills-validate. The install tabs above show the steps for each supported agent.
Which AI agents does skills-validate work with?
It is written for Universal, as a SKILL.md file. Other agents that read the same format can often use it too.
Is skills-validate safe to use?
Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful. It is MIT-licensed and scores 100/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
Is skills-validate still maintained?
The repository was last updated 29 days ago, so skills-validate is actively maintained.

name: skills-validate description: Use when explicitly asked to run a live or plan-only application test for a published Scenario skill. license: MIT metadata: internal: true argument-hint: <skill-name> [--pr <number>] [--plan-only] [--task "..."] [--no-post] [--keep] disable-model-invocation: true

Validate the skill named in the invocation arguments by having a fresh agent do real work with it, then report where that agent got stuck. A defect here is a defect in the skill text, never in the agent.

Flags: --pr <number> targets a PR instead of detecting one, --plan-only runs the zero-cost planning protocol from docs/application-test.md instead of live generation, --task "..." supplies the use case instead of writing one, --no-post stops before publishing anything, --keep keeps the run directory.

Live runs spend Scenario credits. Keep every generation the smallest one that still proves the point. Before step 1, read "Rules learned from past runs" in docs/application-test.md.

Determine the context (step 6) before anything else. A PR from a fork (isCrossRepository: true) is quarantined: follow step 1 of .agents/skills/skills-pr-handle/SKILL.md for what may run, where, and with which tools. Read its skill from refs/pr/$PR, take its mechanical results from the PR's CI, never a local run, and install it in step 4 from git archive "refs/pr/$PR", never the working tree.

1. Review the objective

Read the skill's SKILL.md and every file it links; node scripts/lib/skills.mjs --dir <name> prints its folder, nested under skills/dcc/ or skills/game-engines/ for an expert tool. If no skill has that name, list the close ones and stop.

State, in your own words: the objective (one sentence), the triggering conditions the description claims, and the three to six non-obvious facts the skill exists to teach, the ones an agent would otherwise guess wrong (upload flow, jobs_wait re-calls, runs_as wiring, dry runs, launch semantics). Those facts are the traps the run has to spring.

2. Write a concrete use case

One realistic task, in the words a user would actually use, that forces at least three of those traps and cannot be satisfied by generic MCP intuition (for an expert tool, generic knowledge of the application; its budget counts application runs, not generations). Give it explicit success criteria (which artifacts must exist, and what has to be true of them) and a hard budget (how many generations, which of them may be dry_run). Print the task and the criteria before spending anything. With --task, use the supplied task and still write the criteria.

3. Run the mechanical checks first

They are free and they catch the cheap failures: pnpm spec (spec validation), plus pnpm test when the skill ships a script. Record the results and continue either way; the report carries both layers.

4. Spin up the fresh agent

Build a run directory outside the repository and install the skill under test into it, from the working tree, so the run tests the version under review rather than the published one:

SKILL="<name>"
DIR=$(node scripts/lib/skills.mjs --dir "$SKILL")
RUN=$(mktemp -d "${TMPDIR:-/tmp}/skill-validate-$SKILL-XXXXXX")
mkdir -p "$RUN/.claude/skills" "$RUN/assets"
cp -R skills/scenario "$RUN/.claude/skills/"
case "$DIR" in
  skills/*/*/*) cp -R "${DIR%/*}"/scenario-* "$RUN/.claude/skills/" ;; # expert tool: its whole family
  *) cp -R "$DIR" "$RUN/.claude/skills/" ;;
esac
awk 'f; /^---$/ { if (++c == 2) f = 1 }' .claude/agents/skill-tester.md >"$RUN/contract.md"

Copy scenario alongside every other skill: real installs ship both. An expert tool brings its whole family, since the specialists import the lead skill's scripts/, and a live run needs the application installed: ask which machine and application version the run may drive, and ask for a team and project only when the task hands off to a Scenario skill. Otherwise, ask which team and project the run should use before spending anything: every generation and upload lands in that scope, and the tester must never pick one (teams_list and projects_list enumerate the choices). Write the task from step 2 to $RUN/task.md, including the budget, the success criteria, the run directory path, and the team and project.

Then pick the strongest isolation available. If the Claude Code CLI is installed, run cd "$RUN" && claude mcp list; otherwise use the subagent route below.

  • Separate process (preferred, and a genuinely empty context) when that lists a connected Scenario server:

    cd "$RUN" && claude -p --setting-sources user,project \
      --append-system-prompt "$(cat "$RUN/contract.md")" \
      --allowedTools "mcp__<server> Read Write Bash" \
      --output-format json "$(cat "$RUN/task.md")" | tee "$RUN/result.json"
    

    Use the server name claude mcp list printed, normalized the way tool prefixes are: dots and spaces become underscores, so claude.ai Scenario is mcp__claude_ai_Scenario. The child loads the skills from $RUN/.claude/skills/ and never sees this repository, though --setting-sources user,project (which a user-scope connector needs) also carries the user's plugins and hooks into it; watch the transcript for injected noise. A quarantined fork drops Bash from --allowedTools: its scripts run in the container.

  • Subagent when the CLI is unavailable or the list has no connected Scenario server. Spawn a fresh agent with the contract, the task, and the run directory path (Claude Code can use its skill-tester agent). Isolation is weaker: it can still see repository context, which is why the contract tells it to ignore it. Never for a quarantined fork: skill-tester has a shell.

Say in the report which mode ran. With --plan-only, the same setup applies but the task asks for a numbered tool-call plan with exact tool names and argument shapes, executing nothing.

5. Grade the run

Fetch https://mcp.scenario.com/docs/tools fresh rather than recalling it, then judge. With --plan-only, judge the numbered plan by the same rubric: objective met asks whether the planned calls would reach the criteria, and the evidence table stays empty.

An expert tool is graded against its lead skill's execution channels and the application's own documentation instead of the tool reference: real names are the application's API, command, and UI names for the version the skill targets, the correct flow is the lead's loop (channel choice, stage gates, review renders or captures, audits), and the model-id and MCP bullets below do not apply.

  • Objective met. The artifacts exist and satisfy the criteria from step 2. Open them; do not take the tester's word for it.
  • Real names only. Every tool and parameter the tester used exists. One invented name is a fail.
  • Correct flow. The loop the skill under test teaches. For a generation task that is discovery, model_schema_get, model_run, then jobs_wait re-called with pending_job_ids for any job still running (fast models return complete inline, and job_get polling is never correct), then asset_display or asset_download. Skills built on other loops (workflows, training, reporting) are graded against their own.
  • No constant generative model ids. They came from a recommend step (a capability) or a search step (a member known by name, or private). A named first-party tool model is not a violation when it is a sanctioned singleton or the member the skill under test is dedicated to; a named generative model always is.
  • MCP for the agent's own steps. The calls the tester made itself went through MCP tools. Reaching for the Scenario REST API, the official SDK, or a CLI in place of an MCP tool is a fail even when it produced the artifact, and inventing an API call to cover something MCP does not expose is a worse one: that gap belongs in the defects list. Running a script the skill ships is not a fail, whatever the script uses internally, and neither is saving a URL asset_download returned.
  • Traps handled the way the skill teaches.
  • No guessing. Anything asserted that appears in neither the SKILL.md nor the tool reference is a guess, even when it happens to be right. The tester's guesses and friction entries are the shortest route to the missing sentence.

Verdict: pass, pass with notes, or fail. Tie every defect to the exact line of SKILL.md to add or change. After a fix, re-run with a new fresh agent: an agent that already failed is contaminated by its own mistake.

6. Determine the context

git rev-parse --abbrev-ref HEAD, then look for an open PR with that head branch: gh pr view --json number,url,title,isCrossRepository when gh is available, otherwise the GitHub MCP tools (list_pull_requests with head: <owner>:<branch>). --pr overrides the detection. This runs before step 1. Say which context you found before acting on it.

7. Report

Assemble the report with the template below. This repository is public: only publicly shareable language (see AGENTS.md), never a signed asset URL, which is a credential in itself, and never the team or project the run used. Assets travel as ids and local filenames. A run whose files never passed through Scenario (an expert tool working on local files) drops the Asset id, Job id and Model id columns and says the files were made locally. GitHub has no API for comment attachments, so images reach the thread only when a human drags them into the comment's edit box.

In a PR context, post it as a PR comment (gh pr comment <n> --body-file or the GitHub MCP add_issue_comment), ending with the attribution line, and print the comment URL. --no-post prints the report here instead.

With no PR, ask what to do with it: open a tracker issue for the defects (the scenario-report skill covers the forms and the redaction rules), save it to a file, post it to a PR number they name, or leave it in this conversation. Post nothing until they pick.

Finally, delete the run directory except $RUN/assets/ (keep all of it with --keep). Save there any comparison image you built while grading: it backs a claim in the report.

When $RUN/assets/ holds files, end by recommending which to upload, if any, in a table of file, size, and the report claim it backs:

  • Pick the few that prove the verdict: a contact sheet or the key output, a before and after behind a defect or a gate result, and one short delivered file a reader can play.
  • Leave out inputs, files a smaller pick already proves, and files past GitHub's attachment limits: 10 MB for an image, and 10 MB for a video unless the repository's owner has a paid plan (100 MB).

Then ask whether to open the folder so the files can be dragged into the report comment, and open it (open on macOS, xdg-open on Linux) only on a yes.

After the user says the files are uploaded, offer to format the comment. On a yes:

  1. Fetch its body with gh api repos/<owner>/<repo>/issues/comments/<id> --jq .body. Keep every attachment URL GitHub inserted, and change nothing outside the attachment block.
  2. Move each attachment under the Evidence table, below a heading naming what it shows.
  3. Size each image with an <img width="..."> tag, and replace the alt text GitHub derived from the file name with a description.
  4. Leave each video URL alone on its line, with a one-line caption above it: GitHub renders a bare URL on its own line as a player.
  5. Update the comment with gh api --method PATCH repos/<owner>/<repo>/issues/comments/<id>, passing the new body as -F body=@<file>.
## Skill validation: `<name>` (<verdict>)

Objective: <one sentence>
Use case: <the task, one or two sentences>
Run: <separate process|subagent>, model <model>, commit <sha>, <UTC timestamp>

### Mechanical checks

| Check     | Result |
| --------- | ------ |
| pnpm spec |        |
| pnpm test |        |

### End

Truncated for display — read the full file on GitHub.

Related Skills

View on GitHub
GitHub Stars1.0k
CategoryDevelopment
Updated29d ago
Forks129

Languages

Python

Trust signals

100/100

From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.

No cautions