SkillAgentSearch skills...

evidence-driven-testing

Records visual proof while testing UI behavior — the agent tests the app hands-on via computer use while a screen recording with structured test/assertion annotations captures the session — then posts the video and a results summary to the requested destination

Install / Use

npx skills add michaelshimeles/skills --skill evidence-driven-testing

Installs into whichever agent you are using.

About this skill
📄

SKILL.md

Installable skill definition

Quality Score

86/100

Supported Platforms

Universal

Our assessment of evidence-driven-testing

evidence-driven-testing scores 86/100 on our quality scale, 2439th of 4,698 Development & Engineering skills we index.

Its SKILL.md is 15 KB long, well organised into 14 sections and no code examples: a thorough specification that gives an agent plenty to work with.

With 1,076 GitHub stars, it is one of the more widely adopted skills in the catalogue.

Substance
30/30
Structure
13/20
Description
15/15
Adoption
13/20
Freshness
15/15

Maintenance, license and trust

  • The repository was last updated about 2 months ago, so evidence-driven-testing is actively maintained.
  • Our last check on 2026-11-03 found the source still online.
  • No license is declared. By default that means all rights are reserved: you can read it, but reusing or redistributing it is not clearly permitted. Ask the author before building on it commercially.
  • Its trust signals score 88/100, with 1 caution from licensing, adoption, age or documentation. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.

Safety scan

No issues found

Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful.

AI review by kimi-k2.7-code on 2026-10-07. Automated pattern scan on 2026-10-07. It catches known dangerous patterns, not every risk — read a skill before letting an agent act on it.

evidence-driven-testing compared with similar skills

All 4 of these similar skills score higher than evidence-driven-testing; compare them before choosing.

SkillScoreStarsUpdatedFormat
evidence-driven-testing (this skill)by michaelshimeles861.1k2mo agoSKILL.md
taste-skillby Leonxlnx100116.8k8d agoCLAUDE.md
ai-job-searchby ai-job-search-dev10072.1ktodayCLAUDE.md
claude-howtoby luongnv8910047.3k2d agoCLAUDE.md
lobehubby lobehub10085.6ktodayMCP Server

Frequently asked questions

How do I install evidence-driven-testing?
Run npx skills add michaelshimeles/skills --skill evidence-driven-testing. The install tabs above show the steps for each supported agent.
Which AI agents does evidence-driven-testing work with?
It is written for Universal, as a SKILL.md file. Other agents that read the same format can often use it too.
Is evidence-driven-testing safe to use?
Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful. It declares no license and scores 88/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
Is evidence-driven-testing still maintained?
The repository was last updated about 2 months ago, so evidence-driven-testing is actively maintained.

name: evidence-driven-testing description: > Records visual proof while testing UI behavior — the agent tests the app hands-on via computer use while a screen recording with structured test/assertion annotations captures the session — then posts the video and a results summary to the PR and tracker issue. Use whenever a change needs verifiable evidence that it works, instead of prose claims — including headless environments (scripted screenshots and probes) and non-UI changes (measured numbers, output pairs). compatibility: Screen-recording path requires a GUI environment the agent can drive — built-in computer use, or the cua-driver CLI (trycua/cua) when the harness has no computer-use tools — plus an authenticated browser session for the app under test. The bundled recorder (scripts/evidence.py) runs on Linux (X11 via x11grab, Wayland via wf-recorder), macOS (avfoundation, needs Screen Recording permission) and Windows (gdigrab) and needs Python 3 plus ffmpeg + ffprobe built with libx264 and the ass filter. The headless path requires only a running app and a scriptable browser (e.g. Playwright via npx). Posting evidence requires gh (GitHub CLI) or equivalent. metadata: version: "1.2"

Evidence-Driven Testing

Record annotated proof of behavior, then attach it to the PR and tracker issue.

The recording is the capture of you testing the app via computer use: start the recorder, then drive the app yourself — click, type, navigate — through each test target. Every action in the video is the test being performed live; the recording has no value as evidence unless it shows that interactive session. If the harness has no computer-use tools but a GUI exists, drive the app with cua-driver instead (see below) — it is still your live session.

The bundled recorder, scripts/evidence.py, captures the display with FFmpeg on Linux, macOS, and Windows, timestamps each annotation you add while testing, burns them into the video on stop, and verifies the result with ffprobe. It writes evidence.mp4, report.md, and manifest.json into the session folder.

Inputs

  • Test targets (required): The behaviors/flows to verify, phrased as testable statements.
  • PR / issue (optional): Where to post the evidence. If omitted, deliver to the requester only.

The recorder

EVIDENCE below means the path to scripts/evidence.py inside this skill's folder (wherever the skill is installed, e.g. ~/.claude/skills/evidence-driven-testing/scripts/evidence.py). It needs only Python 3 and FFmpeg.

  • Check first: python3 $EVIDENCE doctor — verifies ffmpeg, ffprobe, libx264, and the ass filter (ready), then which screen-capture source works on this machine (capture_ready and the source auto will pick). Exits non-zero only when the toolchain is missing; read capture_ready before recording.

  • Platforms (--source auto picks the first available):

    | OS | Source | Needs | |---|---|---| | Linux X11 / XWayland | x11 (x11grab) | DISPLAY set | | Linux Wayland | wayland (wf-recorder) | WAYLAND_DISPLAY set, wf-recorder on PATH, and a compositor confirmed to support wlr-screencopy — either a known wlroots one (Sway, Hyprland, river, Wayfire, labwc, dwl, niri) or verified via wayland-info. GNOME and KDE Wayland are not capturable this way; doctor says so. Use x11 through XWayland for X11 apps, or a fallback recorder. --source wayland still forces it | | macOS | avfoundation | Screen Recording permission granted to the terminal / agent host app; doctor lists screen indexes for --screen-index | | Windows | gdigrab | any standard ffmpeg build; powershell for process checks |

    Capture is the full screen by default; --geometry WxH and --offset X,Y crop a region on x11, wayland, and gdigrab. XWayland only sees X11 windows, so prefer the wayland source when the app under test is Wayland-native.

  • Crash-safe raw capture: the raw recording is MPEG-TS (raw.ts), so if the recorder is killed hard or crashes, what was captured still probes and renders. stop remuxes or re-encodes it into a standard evidence.mp4.

  • How stopping works: on Linux the recorder is signalled through a pidfd. On macOS and Windows start launches a small supervisor process that owns the ffmpeg child, and stop asks it (via stop.request in the session folder) to interrupt, then terminate, then kill; it writes recorder-exit.json when done. Nothing ever signals a bare PID, so a recycled PID can't be hit. If the supervisor dies while the recorder is still running, stop refuses and tells you which PID to stop by hand. If it died before it could even record which process it started, stop stays blocked until you have checked for a stray recorder yourself and rerun it with --accept-untracked-recorder; the report then carries that caveat.

  • Fallback recorders when doctor reports no capture source (for example Wayland without wf-recorder): cua-driver recording start <dir> / stop (see the cua-driver section), or the OS recorder (macOS: screencapture -v out.mov). On these paths there is no annotation overlay, so keep the annotation protocol as files — an assertions.md listing each setup / test_start / assertion with its result and the approximate video timestamp, exactly as in the headless path.

  • Never present --source test (the synthetic pattern generator) as UI evidence. It exists to smoke-test the toolchain; the repo's tests/test_evidence.py exercises it.

Instructions

1. Prepare the screen

  • Maximize the browser/app window; close popups, notifications, and extra panels.
  • Navigate to the starting state (logged in, correct page) BEFORE recording, unless setup itself is under test.
  • Note the exact revision under test: git rev-parse HEAD and git branch --show-current (or the deployment URL) — the recorder stamps them into the report.

2. Start recording

  • Begin the screen recording before the first meaningful action:

    python3 $EVIDENCE start \
      --output .artifacts/<task-name> \
      --title "<what is being verified>" \
      --commit "$(git rev-parse HEAD)" --branch "$(git branch --show-current)" \
      --environment "<OS / browser / display / deployment>"
    

    It prints JSON with a session path and the chosen source; keep the path (SESSION=...) for every later command. The source is auto-detected and the whole screen is captured; pass --source, --geometry, --offset, --display/--xauthority (X11), --screen-index (macOS), or --output-name (Wayland) only when doctor or the situation calls for it.

  • Add a setup annotation describing the starting context:

    python3 $EVIDENCE annotate "$SESSION" --type setup \
      --message "Logged in, navigating to connectors page"
    

3. Test via computer use, annotating as you go

  • Perform every interaction through computer use on the live app — the recording captures your session, so the testing and the evidence are the same act. Work at a watchable pace: let the UI settle after each action so state changes are visible on video.

  • At each named test's start, add a test_start annotation in Jest style:

    python3 $EVIDENCE annotate "$SESSION" --type test_start \
      --message "It should execute the tool directly when permission is 'always'"
    
  • After each check, add an assertion annotation with --result passed, failed, or untested:

    python3 $EVIDENCE annotate "$SESSION" --type assertion --result passed \
      --message "Tool ran without a permission prompt"
    
  • Rules for assertions:

    • One assertion per meaningful state change — consolidate, don't annotate per UI label.
    • Use "Precondition: ..." assertions to establish starting state.
    • Keep under 80 characters, high-signal (the recorder rejects longer messages).
    • If a test cannot run (missing prerequisite, expired auth window), mark it untested with the reason — never skip silently.
    • The timestamp records when you asserted, not whether it was true — look at the screen before choosing passed.

4. Stop and review

  • Stop recording after the final assertion:

    python3 $EVIDENCE stop "$SESSION"
    

    This stops the capture (gracefully, so the recorder flushes; escalating only if it ignores the request), burns the annotations into evidence.mp4, probes the result, and writes report.md and manifest.json next to it. It prints "verified": true on success; if rendering fails the session is marked finalization_failed — fix the reported cause and run stop again (a retry does not signal the recorder twice). If the recorder process can no longer be signalled safely (it died, or its PID now belongs to another process), the session is marked recorder_lost. Running stop again finalizes whatever video was captured, but only once that recorder process is confirmed gone — if it is still alive, stop it first, or the video would be rendered while still being written.

  • Confirm the recording captured the key moments before sharing: extract a frame at each assertion timestamp (ffmpeg -ss <t> -i evidence.mp4 -frames:v 1 frame.png) and check the state and the label are visible.

  • Fill in the Caveats section of report.md; never leave the placeholder.

5. Post the evidence

  • report.md is the report: what was tested, environment + exact commit, pass/fail per test, caveats. Extend it rather than rewriting from scratch.
  • Post the video + summary as a PR comment (embed in the PR description if it's your PR). gh pr comment cannot attach a local video — upload evidence.mp4 through the PR's comment box in an authenticated browser, or upload it to a host and link it (for example the before-and-after upload adapters). Reopen the comment and confirm the video plays before claiming it is posted.
  • Attach the same video to the tracker issue (Linear/Jira) with a one-line result.
  • Send the report + recording to the requester.

Guardrails

  • The video must show the actual test session being driven live. Never present scripted playback, stitched clips, or synthetic footage as a recording; if the harness lacks computer-use tools but a GUI exists, drive via cua-driver; with no GUI at all, use the headless path instead.
  • Never record a half-covered or tiled window — maximize first.
  • Never record a screen showing secrets, tokens, customer data, or payment details; if a flow requires them, mark it untested and say why.
  • When verifying a fix, show or reference the old failure alongside the new success.
  • Always state the exact commit/branch/deployment tested against.

No computer-use tools? Drive with cua-driver (GUI available)

When a display exists but the agent has no built-in computer-use capability, use cua-driver (macOS / Windows / Linux) as the actuator. It is still you testing the app live — the recording rule holds unchanged; only the input mechanism differs.

  • Verify the setup with cua-driver doctor before recording. If a cua-driver skill is installed, read it and follow its protocol — the snapshot-before-action invariant is mandatory.
  • Loop per interaction: launch_app → get_window_state (accessibility tree
    • screenshot) → act via element_token (click, type_text, press_key) → verify_state for the expected postcondition. Each verify_state check maps 1:1 onto an assertion annotation.
  • Wherever doctor reports capture_ready: yes, keep using the bundled recorder above for the video and the annotations; cua-driver only supplies the input.
  • Otherwise, cua-driver recording start <output-dir> / cua-driver recording stop is the recorder (the output directory is required, and the daemon must be running: cua-driver serve). Video capture is on by default and is fina

Truncated for display — read the full file on GitHub.

Related Skills

View on GitHub
GitHub Stars1.1k
CategoryDevelopment
Updated2mo ago
Forks181

Languages

Python

Trust signals

88/100

From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.

1 medium
evidence-driven-testing — Universal Skill: Install & Safety Check | SkillAgent