---

name: media-judge description: >- Judge a finished AI-made video, image or track by measuring it, then rank takes on the defects found. One command reads every frame and writes facts with timestamps and evidence: broken frames, banding after a simulated X or YouTube upload, numbers that jiggle, spelling, too much on screen, nothing leading the eye, motion with no cause, pinned graphics, sliding labels, a face, outfit or light that changed, sound after the cut, loudness, voice against the script, and distance to references against the template. A fresh agent settles only what eyes can settle. No score out of 10. Use when someone says "review this video", "check my render", "self-review", "QA", "judge", "critique", "score these takes", "pick the best take", "does it match the reference", "it looks like a template", "is it ready to post", "reverse engineer this reel", or after any agent finishes a film, short drama, reel or ad. Not for making or repairing media (varg-ai, varg-video-editing) or ad law (ai-ad-compliance-check). license: MIT metadata: author: vargHQ version: "1.0.0" openclaw: requires: bins: - ffmpeg - ffprobe - uv homepage: https://github.com/vargHQ/skills/tree/main/media-judge compatibility: >- Needs ffmpeg built with libvmaf (the banding test; brew's build has it) and uv, which installs OpenCV, NumPy, RapidOCR and a spell checker on first run. Runs on a laptop CPU with no API key and no credits. Downloads two small OpenCV face models from Hugging Face on first use. Voice against a script needs faster-whisper (uv run --with faster-whisper). allowed-tools: Bash(uv:) Bash(ffmpeg:) Bash(ffprobe:*) Read Write

Media judge

Measure, don't grade. A judgment here is a time, a number and a picture. "Looks great", "8/10" and "feels premium" are not findings, and neither is any adjective with no measurement behind it.

Code finds the moment and a fresh pair of eyes reads only that moment, in that order, because the other way round fails: 12 vision models scored at most 54.1% on spotting temporal glitches; Claude Opus caught 17.5% of them, Gemini 3 Flash said "glitch" to about 87% of clean clips (TempGlitch, gameplay clips). Uniform sampling at 0.5 fps caught 11.81% of faults under 0.2 s (SynFlash), and GPT-4 preferred its own output 0.705 and 0.912 of the time on tasks where people saw equal quality (Panickssery et al.). So:

1. Facts: judge.py reads every frame and every sample. It does not sample.
2. Eyes: a fresh agent opens only the evidence and answers narrow questions.
3. Verdict: counted defects. Ranking follows from the counts, never from a feeling.

What this has been tested on. Planted defects in footage it never saw while being tuned: 18 of 22 caught, 6 of 8 clean controls (25 of 25 and 10 of 10 on the footage it was tuned on). In a blind test on 19 anonymous clips, an agent with this skill found 12 of 14 planted defects, with no false claims on clean clips, using 116k tokens. An agent given only the failure list found 11 of 14, using 197k tokens, and reported the banding without an upload test. So the list itself does a lot. This adds repeatable numbers, the upload test, and fewer tokens. Both missed a face swap they weren't told to look for, so use --one-character.

1. Who judges

If you made this media, you do not judge it in this context. Start a fresh agent with no knowledge of the prompts or of how it was made, and give it only the file, this skill, the brief (what it must say, names, product) and the references. If you cannot start one, write self-judged at the top of the verdict.

2. Run the measurements

uv run scripts/judge.py video film.mp4 --out runs/film \
  --allow "varg,Seedance"          # brand and product names for the spell check

Takes about a minute for a 12 s clip and about 5 minutes for a 47 s 1080p film on a busy laptop. Add what you know, because each flag turns a report into a test:

| Flag | Use it when | What it turns on | |---|---|---| | --one-character | the brief says one person, or the same person across shots | face changes become flags. Without it a swap is only a number in neighbour_pairs, and a blind run of this skill missed one that way | | --groups '[[0,1,2],[5,6]]' | these shots share one scene | outfit and light jumps become flags | | --expected lines.txt | you know the on-screen text | every line must appear | | --script voice.txt | there is a voice-over | word error rate, with --with faster-whisper | | --refs ref.mp4 | it copies a reference | pair sheet and distance per shot | | --template tpl.mp4 | it was built from a template | margin: closer to references or to template | | --labels labels.json | you have the render's anchors | exact label drift against its target | | --platform x\|youtube\|reels\|tiktok\|shorts | you know where it goes | upload simulation and safe zones | | --size 1080x1920 | a size was requested | size becomes a blocker | | --music-only | no voice in the piece | skips voice tests |

Images and tracks: judge.py image poster.png --out runs/poster [--ref ref.png] and judge.py audio track.mp3 --out runs/track.

Exit code 0 means nothing was flagged, 1 means flags to look at, 2 means the tool failed.

3. Read the report

Open runs/film/report.md. Each check is in one of four states:

• flagged: over its threshold. Open the evidence before you believe it.
• clear: measured and under the threshold. It says nothing beyond that one measurement.
• info: a number with no pass or fail, usually because nothing could be tested (no moving shot to test pinning, no counter on screen).
• not_run: a tool or model was missing.

A clear report does not mean a good video. "Not settled by this run" at the bottom of the report lists what is still open. references/checks.md explains each check: how it works, its threshold and its source, what false flags look like. Read the entry before overturning or trusting a flag on a check you have not used before.

Expect to overturn flags. On a real 47 s film we overturned all 4 spelling candidates we opened (OCR misreading small interface text), a subtitle measured while fading in, and a keyboard read as a pinned graphic. The upload banding and the unreadable screens were real.

4. Use your eyes on what code flagged

Work through questions in runs/film/facts.json. Each one names the frames to open and one narrow question. Read references/eyes.md first. The rules that change results:

• Open contact_sheet_*.jpg and each flagged evidence file. Do not judge from memory of the prompt.
• Ask "describe" and "what is unusual", never "what is wrong" (57.2% against 29.5% on GlitchBench).
• Answer per shot, with yes or no, a time, and something you can quote from the frame to prove you looked.
• Do not estimate where people look. Models score about 0.34 to 0.41 against real gaze. Use the heat map.
• Ask for observations, not verdicts, and let code decide (references/observe-prompt.txt). Never ask "which is better?" once: our judge picked the take shown second 75% of the time.
• A hunch without evidence goes under "Hints", never in the defect list.

Code cannot settle these, so eyes decide, using the measured moments: whether a move has a cause, whether a pinned graphic should be pinned, whether the most salient thing is the right thing, whether an outfit or light change is intended, and whether it looks like the references or just near them.

5. Challenge it

One pass, then stop (three at most).

• For each finding: what would make this not a defect? Is it in the brief? Is it OCR misreading a word? Is the "late sound" an unrelated noise near a cut? Open the frames again at source size. Keep the finding, or overturn it and write why.
• For each clear that matters: open one example. A check that stops firing looks exactly like a clean video.
• Stranger test: can you name three things in these frames that could only come from this brief? If not, report how many you found.
• Re-measure after a fix. A fix is not done until the same command stops flagging it, and nothing new appears.

6. Write the verdict

# Verdict: film.mp4
self-judged: no · ran 27 checks, 21 clear · not settled: lip sync, voice script

State: stopped (2 blockers)

| Severity | Time | Where | Measured | Evidence | Fix | Cost |
|---|---|---|---|---|---|---|
| blocker | 00:05.00 | whole frame | 2 black frames between lit frames | evidence/black_120.jpg | trim 2 frames at the seam | edit |
| major | 00:06-00:10 | dark gradient | CAMBI 4.1 source, 9.8 after upload at 2 Mbps | evidence/banding_3.jpg | add grain before encoding, or lift blacks | edit |

## Eyes
Shot 3, 00:07.20: label "Battery" points at the cell in the first frame and sits 90 px left of it in the last.

## Hints (no measurement behind these)
...

## Not checked
Mispronounced names. Whether the real X encoder matches the simulation.

Every row has a time, a number, a file and a fix. Cost is cut, edit or retake. Fix the cheapest first and name what stays as it is. Severity comes from SEVERITY in scripts/mj_rank.py. Change it there if a piece needs different rules, and say so.

Then save what eyes settled in runs/film/overrides.json, so it counts in the ranking and the reason stays on record:

{"overturned": {"spelling": "all three are OCR misreads, checked each crop"},
 "eyes": [{"severity": "major", "t": 7.2, "what": "label Battery no longer touches the cell"}]}

The score is the count. State is stopped with any blocker, needs a look with a major flag, no measured defect otherwise. ran 27, 21 clear is the only number out of something. Do not blend these into one mark.

7. Rank takes

uv run scripts/judge.py rank runs/take1 runs/take2 runs/take3 --out runs/ranking

The order is blockers, then major flags, then minor flags, then the reference margin. ranking.md says why each take beats the next, and it counts each run's overrides.json. For takes tied on facts, judge each take alone with observations and let code decide, then use pairs in both orders where a flip stays a tie. See references/llm-judge.md for the protocol and how to write criteria.

8. Compare with a reference or a template

--refs aligns shots by picture and writes evidence/reference_pairs_0.jpg, ours next to theirs, worst pairs first, with distances for structure, palette, composition, texture and tone, plus shot-length rhythm. Open the sheet. Numbers say where it differs. Eyes say whether that matters.

With both --refs and --template you get margin = distance to template - distance to references. Above 0 means closer to the references. These distances are hand-built on OpenCV features. No study validates them for taste, so treat a small margin as a tie and look at the sheet.

9. Reverse engineer a finished piece

uv run scripts/judge.py reverse ad.mp4 --out runs/ad

This writes a shot list from measurement: start, length, camera move and speed, shake, palette, lightness, where the focus sits, on-screen text, sound events, tempo, loudness, cuts per 10 s. Then open the sheets and add what code cannot measure: the hook, the lens, the light, the intent. Mark those as estimates. Nothing recovers the prompt or the seed. Say so.

10. Keep the judge honest

• tests/make_fixtures.py plants one known defect per clip in real footage, and tests/run_bench.py reports catches, misses and false flags. Run it after changing a check. Add a fixture every time a person finds something the judge missed.
• scripts/thresholds.json holds every threshold with status: published, convention or calibrated_on_fixtures. Only a few are published. Calibrate the