evidence-driven-testing
Records visual proof while testing UI behavior — the agent tests the app hands-on via computer use while a screen recording with structured test/assertion annotations captures the session — then posts the video and a results summary to the requested destination
Install / Use
npx skills add michaelshimeles/skills --skill evidence-driven-testingInstalls into whichever agent you are using.
SKILL.md
Installable skill definition
Quality Score
Category
Development & EngineeringSupported Platforms
Our assessment of evidence-driven-testing
evidence-driven-testing scores 86/100 on our quality scale, 2439th of 4,698 Development & Engineering skills we index.
Its SKILL.md is 15 KB long, well organised into 14 sections and no code examples: a thorough specification that gives an agent plenty to work with.
With 1,076 GitHub stars, it is one of the more widely adopted skills in the catalogue.
Maintenance, license and trust
- The repository was last updated about 2 months ago, so evidence-driven-testing is actively maintained.
- Our last check on 2026-11-03 found the source still online.
- No license is declared. By default that means all rights are reserved: you can read it, but reusing or redistributing it is not clearly permitted. Ask the author before building on it commercially.
- Its trust signals score 88/100, with 1 caution from licensing, adoption, age or documentation. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.
Safety scan
No issues foundOur scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful.
AI review by kimi-k2.7-code on 2026-10-07. Automated pattern scan on 2026-10-07. It catches known dangerous patterns, not every risk — read a skill before letting an agent act on it.
evidence-driven-testing compared with similar skills
All 4 of these similar skills score higher than evidence-driven-testing; compare them before choosing.
| Skill | Score | Stars | Updated | Format |
|---|---|---|---|---|
| evidence-driven-testing (this skill)by michaelshimeles | 86 | 1.1k | 2mo ago | SKILL.md |
| taste-skillby Leonxlnx | 100 | 116.8k | 8d ago | CLAUDE.md |
| ai-job-searchby ai-job-search-dev | 100 | 72.1k | today | CLAUDE.md |
| claude-howtoby luongnv89 | 100 | 47.3k | 2d ago | CLAUDE.md |
| lobehubby lobehub | 100 | 85.6k | today | MCP Server |
Frequently asked questions
- How do I install evidence-driven-testing?
- Run
npx skills add michaelshimeles/skills --skill evidence-driven-testing. The install tabs above show the steps for each supported agent. - Which AI agents does evidence-driven-testing work with?
- It is written for Universal, as a SKILL.md file. Other agents that read the same format can often use it too.
- Is evidence-driven-testing safe to use?
- Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful. It declares no license and scores 88/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
- Is evidence-driven-testing still maintained?
- The repository was last updated about 2 months ago, so evidence-driven-testing is actively maintained.
Skill content
View source on GitHubname: evidence-driven-testing description: > Records visual proof while testing UI behavior — the agent tests the app hands-on via computer use while a screen recording with structured test/assertion annotations captures the session — then posts the video and a results summary to the PR and tracker issue. Use whenever a change needs verifiable evidence that it works, instead of prose claims — including headless environments (scripted screenshots and probes) and non-UI changes (measured numbers, output pairs). compatibility: Screen-recording path requires a GUI environment the agent can drive — built-in computer use, or the cua-driver CLI (trycua/cua) when the harness has no computer-use tools — plus an authenticated browser session for the app under test. The bundled recorder (scripts/evidence.py) runs on Linux (X11 via x11grab, Wayland via wf-recorder), macOS (avfoundation, needs Screen Recording permission) and Windows (gdigrab) and needs Python 3 plus ffmpeg + ffprobe built with libx264 and the ass filter. The headless path requires only a running app and a scriptable browser (e.g. Playwright via npx). Posting evidence requires gh (GitHub CLI) or equivalent. metadata: version: "1.2"
Evidence-Driven Testing
Record annotated proof of behavior, then attach it to the PR and tracker issue.
The recording is the capture of you testing the app via computer use: start the
recorder, then drive the app yourself — click, type, navigate — through each
test target. Every action in the video is the test being performed live; the
recording has no value as evidence unless it shows that interactive session.
If the harness has no computer-use tools but a GUI exists, drive the app with
cua-driver instead (see below) — it is still your live session.
The bundled recorder, scripts/evidence.py, captures the display with FFmpeg
on Linux, macOS, and Windows, timestamps each annotation you add while
testing, burns them into the video on stop, and verifies the result with
ffprobe. It writes evidence.mp4, report.md, and manifest.json into the
session folder.
Inputs
- Test targets (required): The behaviors/flows to verify, phrased as testable statements.
- PR / issue (optional): Where to post the evidence. If omitted, deliver to the requester only.
The recorder
EVIDENCE below means the path to scripts/evidence.py inside this skill's
folder (wherever the skill is installed, e.g.
~/.claude/skills/evidence-driven-testing/scripts/evidence.py). It needs only
Python 3 and FFmpeg.
-
Check first:
python3 $EVIDENCE doctor— verifiesffmpeg,ffprobe,libx264, and theassfilter (ready), then which screen-capture source works on this machine (capture_readyand the sourceautowill pick). Exits non-zero only when the toolchain is missing; readcapture_readybefore recording. -
Platforms (
--source autopicks the first available):| OS | Source | Needs | |---|---|---| | Linux X11 / XWayland |
x11(x11grab) |DISPLAYset | | Linux Wayland |wayland(wf-recorder) |WAYLAND_DISPLAYset,wf-recorderon PATH, and a compositor confirmed to support wlr-screencopy — either a known wlroots one (Sway, Hyprland, river, Wayfire, labwc, dwl, niri) or verified viawayland-info. GNOME and KDE Wayland are not capturable this way;doctorsays so. Usex11through XWayland for X11 apps, or a fallback recorder.--source waylandstill forces it | | macOS |avfoundation| Screen Recording permission granted to the terminal / agent host app;doctorlists screen indexes for--screen-index| | Windows |gdigrab| any standard ffmpeg build;powershellfor process checks |Capture is the full screen by default;
--geometry WxHand--offset X,Ycrop a region on x11, wayland, and gdigrab. XWayland only sees X11 windows, so prefer thewaylandsource when the app under test is Wayland-native. -
Crash-safe raw capture: the raw recording is MPEG-TS (
raw.ts), so if the recorder is killed hard or crashes, what was captured still probes and renders.stopremuxes or re-encodes it into a standardevidence.mp4. -
How stopping works: on Linux the recorder is signalled through a pidfd. On macOS and Windows
startlaunches a small supervisor process that owns the ffmpeg child, andstopasks it (viastop.requestin the session folder) to interrupt, then terminate, then kill; it writesrecorder-exit.jsonwhen done. Nothing ever signals a bare PID, so a recycled PID can't be hit. If the supervisor dies while the recorder is still running,stoprefuses and tells you which PID to stop by hand. If it died before it could even record which process it started,stopstays blocked until you have checked for a stray recorder yourself and rerun it with--accept-untracked-recorder; the report then carries that caveat. -
Fallback recorders when
doctorreports no capture source (for example Wayland without wf-recorder):cua-driver recording start <dir>/stop(see the cua-driver section), or the OS recorder (macOS:screencapture -v out.mov). On these paths there is no annotation overlay, so keep the annotation protocol as files — anassertions.mdlisting eachsetup/test_start/assertionwith its result and the approximate video timestamp, exactly as in the headless path. -
Never present
--source test(the synthetic pattern generator) as UI evidence. It exists to smoke-test the toolchain; the repo'stests/test_evidence.pyexercises it.
Instructions
1. Prepare the screen
- Maximize the browser/app window; close popups, notifications, and extra panels.
- Navigate to the starting state (logged in, correct page) BEFORE recording, unless setup itself is under test.
- Note the exact revision under test:
git rev-parse HEADandgit branch --show-current(or the deployment URL) — the recorder stamps them into the report.
2. Start recording
-
Begin the screen recording before the first meaningful action:
python3 $EVIDENCE start \ --output .artifacts/<task-name> \ --title "<what is being verified>" \ --commit "$(git rev-parse HEAD)" --branch "$(git branch --show-current)" \ --environment "<OS / browser / display / deployment>"It prints JSON with a
sessionpath and the chosensource; keep the path (SESSION=...) for every later command. The source is auto-detected and the whole screen is captured; pass--source,--geometry,--offset,--display/--xauthority(X11),--screen-index(macOS), or--output-name(Wayland) only whendoctoror the situation calls for it. -
Add a
setupannotation describing the starting context:python3 $EVIDENCE annotate "$SESSION" --type setup \ --message "Logged in, navigating to connectors page"
3. Test via computer use, annotating as you go
-
Perform every interaction through computer use on the live app — the recording captures your session, so the testing and the evidence are the same act. Work at a watchable pace: let the UI settle after each action so state changes are visible on video.
-
At each named test's start, add a
test_startannotation in Jest style:python3 $EVIDENCE annotate "$SESSION" --type test_start \ --message "It should execute the tool directly when permission is 'always'" -
After each check, add an
assertionannotation with--result passed,failed, oruntested:python3 $EVIDENCE annotate "$SESSION" --type assertion --result passed \ --message "Tool ran without a permission prompt" -
Rules for assertions:
- One assertion per meaningful state change — consolidate, don't annotate per UI label.
- Use "Precondition: ..." assertions to establish starting state.
- Keep under 80 characters, high-signal (the recorder rejects longer messages).
- If a test cannot run (missing prerequisite, expired auth window), mark it
untestedwith the reason — never skip silently. - The timestamp records when you asserted, not whether it was true — look at
the screen before choosing
passed.
4. Stop and review
-
Stop recording after the final assertion:
python3 $EVIDENCE stop "$SESSION"This stops the capture (gracefully, so the recorder flushes; escalating only if it ignores the request), burns the annotations into
evidence.mp4, probes the result, and writesreport.mdandmanifest.jsonnext to it. It prints"verified": trueon success; if rendering fails the session is markedfinalization_failed— fix the reported cause and runstopagain (a retry does not signal the recorder twice). If the recorder process can no longer be signalled safely (it died, or its PID now belongs to another process), the session is markedrecorder_lost. Runningstopagain finalizes whatever video was captured, but only once that recorder process is confirmed gone — if it is still alive, stop it first, or the video would be rendered while still being written. -
Confirm the recording captured the key moments before sharing: extract a frame at each assertion timestamp (
ffmpeg -ss <t> -i evidence.mp4 -frames:v 1 frame.png) and check the state and the label are visible. -
Fill in the Caveats section of
report.md; never leave the placeholder.
5. Post the evidence
report.mdis the report: what was tested, environment + exact commit, pass/fail per test, caveats. Extend it rather than rewriting from scratch.- Post the video + summary as a PR comment (embed in the PR description if
it's your PR).
gh pr commentcannot attach a local video — uploadevidence.mp4through the PR's comment box in an authenticated browser, or upload it to a host and link it (for example thebefore-and-afterupload adapters). Reopen the comment and confirm the video plays before claiming it is posted. - Attach the same video to the tracker issue (Linear/Jira) with a one-line result.
- Send the report + recording to the requester.
Guardrails
- The video must show the actual test session being driven live. Never present
scripted playback, stitched clips, or synthetic footage as a recording; if
the harness lacks computer-use tools but a GUI exists, drive via
cua-driver; with no GUI at all, use the headless path instead. - Never record a half-covered or tiled window — maximize first.
- Never record a screen showing secrets, tokens, customer data, or payment
details; if a flow requires them, mark it
untestedand say why. - When verifying a fix, show or reference the old failure alongside the new success.
- Always state the exact commit/branch/deployment tested against.
No computer-use tools? Drive with cua-driver (GUI available)
When a display exists but the agent has no built-in computer-use capability, use cua-driver (macOS / Windows / Linux) as the actuator. It is still you testing the app live — the recording rule holds unchanged; only the input mechanism differs.
- Verify the setup with
cua-driver doctorbefore recording. If acua-driverskill is installed, read it and follow its protocol — the snapshot-before-action invariant is mandatory. - Loop per interaction:
launch_app→get_window_state(accessibility tree- screenshot) → act via
element_token(click,type_text,press_key) →verify_statefor the expected postcondition. Eachverify_statecheck maps 1:1 onto anassertionannotation.
- screenshot) → act via
- Wherever
doctorreportscapture_ready: yes, keep using the bundled recorder above for the video and the annotations; cua-driver only supplies the input. - Otherwise,
cua-driver recording start <output-dir>/cua-driver recording stopis the recorder (the output directory is required, and the daemon must be running:cua-driver serve). Video capture is on by default and is fina
Truncated for display — read the full file on GitHub.
Related Skills
taste-skill
116.8kAgent skills that give coding agents taste: art direction, reference images and a browser check instead of generic AI pages
ai-job-search
72.1kThe job search that runs on your machine. AI job application framework built on Claude Code: evaluate postings, tailor CVs, write cover letters, prep interviews. Fork it and own it.
claude-howto
47.3kA visual, example-driven guide to Claude Code — from basic concepts to advanced agents, with copy-paste templates that bring immediate value.
lobehub
85.6k🤯 LobeHub - Super-Agents on Your Team. Add super-agents to the chats your team already uses.They take on work by themselves and get better every week.
Languages
Trust signals
From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.
