---

name: varg-music-video description: >- Directs an AI music video or a song-driven promo end to end on varg, the way the most-watched AI music videos are made: measure the song (beat grid, word timings), pitch the concept, cast and lock character references, storyboard every cut on a downbeat, approve stills before paying for video, generate Seedance 2.5 shots, run a mandatory lip-sync pass, draw lyrics and graphics in code, run review rounds and master for each platform. Use when someone asks for a music video, a clip for my song, a video for a track, a lyric video with AI footage, a singing character, an AI artist or band video, slopcore, or a launch video cut to a song. Not for a one-off generated song or single clip (use varg-ai), shot planning and 3D camera blocking only (use varg-previz), a talking character without music (use varg-character-keynote or varg-podcast-video), or a typography-only lyric video with no generated footage (use a HyperFrames music-to-video skill). license: MIT metadata: author: vargHQ version: "1.0.0" openclaw: homepage: https://varg.ai compatibility: >- Generation needs the varg MCP connection (https://mcp.varg.ai/mcp) or a VARG_API_KEY (see the varg-ai skill). Song analysis uses local tools when available: ffmpeg, Python with Demucs and a beat tracker. Code-drawn lyrics and graphics need HyperFrames (Node 22 + ffmpeg) or Remotion. allowed-tools: Bash(curl:) Bash(ffmpeg:) Bash(ffprobe:) Bash(python3:) Bash(bun:) Bash(npx:) Read Write Edit

Music video director

This skill encodes the workflow that the most-watched AI music videos on X in 2026 actually use (sources at the end): a long slopcore guide written after four viral videos, a character series with a 20M-view video, a Suno → Seedance thread, a Seedance 2.5 reference-bible guide, and a restyle-a-reference-video breakdown.

It is written as a capability pipeline. Each step names the capability it needs and the tool that provides it: a varg model or tool where one exists, otherwise an outside tool marked Outside varg. Do not drop a step because varg lacks a tool for it. Use the outside tool, tell the user what it needs, or stop at that gate.

Generation (images, video, voice, music, lip-sync, transcription, ffmpeg) runs on varg: the varg MCP server (https://mcp.varg.ai/mcp) or the varg-ai skill (REST at https://api.varg.ai/v2). Read varg-ai for sign-in and model details.

The agent is the crew; the user is the director. Every phase ends at a gate where the user looks at something and says yes before more credits are spent.

A worked example with file formats and ready scripts (song map, frame-exact rough cut, QA contact sheet, small export) is in references/anime-case-study.md: a 99 s, 36-shot anime music video and the fixes from its nine review rounds.

---

0. Non-negotiables (what every good creator converged on)

1. The song is the clock. Measure every beat and every sung word before making a single picture. Every cut and every lyric hangs off those numbers.
2. Cuts land on downbeats. Always. A cut that misses the "one" feels sloppy even to viewers who can't say why. The edit script must refuse off-grid cuts.
3. Fix stills, not clips. A still costs a few credits; a clip costs tens to hundreds, and the video model keeps every fault in the still. If two takes miss the same thing, stop paying for takes and change the still.
4. Everything is video. A slow zoom on a still reads as a slideshow. Budget for real clips.
5. Video models never draw text. End every video prompt with "No text, letters or logos." All lyrics, titles, UI, scoreboards and labels are drawn in code on top of the footage.
6. Seedance re-sings your line in its own timing. Every singing or talking shot gets a dedicated lip-sync pass against the real vocal. It costs less than one retake.
7. Few singing shots. Decide the count in the storyboard and keep it small: 8 of 93 shots in one viral video, 2 in another.
8. Pilot, quote, "Go". Run one clip of each shot kind, show the result and an exact price in credits, and wait for the user to type go. Every batch has a spending ceiling.
9. When the user criticizes something, make it better. Never remove a feature because of a complaint; ask before cutting anything.
10. Write the user's own words into the brief verbatim. Every later agent reads them.

First video ever? Do the best 30 seconds of the song (usually the first chorus), about 8 shots, one lead character, at most one singing shot, and a hard credit ceiling agreed up front (see section 12).

---

1. Measure the song (gate: timing proof)

Input. The WAV/MP3 plus the lyric sheet. If there is no song yet, see section 1a.

Steps.

1. Split stems into vocals, drums, bass and other.
   • Outside varg: Demucs (pip install demucs, then demucs <song>), run locally.
2. Fit a beat grid to the kick drum.
   • Sanity-check the BPM against the drum stem: a half-time or double-time read (80 instead of 120) puts every cut in the wrong place.
   • If a song has drops, build the grid in sections that reset on the first kick after each drop; a single grid drifts.
   • Record downbeats, bars, sections and drops.
   • Outside varg: a beat tracker such as librosa or madmom on the drum stem. A ready script that fits the tempo per 8 s window and finds the bar phase: songmap.py (references/anime-case-study.md).
3. Get word timings.
   • Run Whisper on the vocal stem, with the lyric sheet passed as the prompt.
   • Align the lyric sheet to what Whisper heard, giving a start and end for every sung word.
   • Whisper folds pauses into the next word, so move each word start to the vocal onset (where the vocal stem gets loud).
   • varg: the transcription tool returns word-level timestamps (whisper_large_v3, or groq_whisper_large_v3_turbo for speed); upload the stem with POST /v2/files first. Outside varg: forced alignment to the lyric sheet and onset correction (a forced aligner such as WhisperX, plus an onset detector on the vocal stem).
4. Render a timing proof.
   • A plain video in which each word lights up at its measured time over a visible beat/bar counter, with the master audio.
   • The user watches it on headphones; iterate until they say it's perfect, then freeze the numbers.
   • It takes minutes; never skip it.
   • Build it with the same code-graphics engine as section 8 (HyperFrames or Remotion).

Output: songmap.json.

{
  "bpm": 98, "grid_segments": [{ "from": 0.0, "bpm": 98, "first_downbeat": 0.21 }],
  "beats": [0.21, 0.82, ...], "downbeats": [0.21, 2.66, ...],
  "sections": [{ "name": "chorus1", "start": 42.1, "end": 61.7 }],
  "drops": [61.7],
  "words": [{ "w": "light", "start": 44.02, "end": 44.51, "line": 12 }],
  "lines": [{ "i": 12, "text": "I keep on waiting for the light", "start": 43.4, "end": 45.9 }],
  "duration": 229.4, "stems": { "vocals": "...", "drums": "..." }
}

1a. No song yet

The creators' default:

1. Lyrics: written with an LLM by the director.
2. Song: Suno, where the artist's persona voice lives. Outside varg: Suno (persona voice and stem export).

On varg:

• lyria_3_5: full song with vocals, up to ~3 min; timestamped structure in the prompt; ≤1000 characters.
• eleven_music_v2_5: songs with vocals, 3 s to 10 min.

State an exact BPM and structure in the prompt either way.

Keep one voice per artist across songs. For spoken lines, creators prefer Fish Audio clones for acting quality, or generate the lines in Suno so speech and singing share one voice. On varg, spoken lines use ElevenLabs (eleven_v4, voice clones supported).

---

2. Pitch the concept and the lyric surfaces (gate: brief approved)

1. Pitch in a few sentences. Write down the joke or the feeling of the song. Let the agent pitch 2–3 concepts and the user react.

2. Give every lyric a home, a surface that belongs to the world of the song. Examples:

   • a nutrition label for the chorus
   • a code review
   • a trading screen
   • phone texts
   • a mission-control HUD
   • hand-painted shop signs

3. Straight-face rule for funny songs. The graphics present the joke as real information, the way a label or news ticker would.

4. Write the production brief to brief.md. It holds:

   • the concept
   • the lyric surfaces
   • the look: palette, era, lens
   • a banned list (e.g. decorative shadows, gradients, emoji, lens flares, glitch effects)
   • the user's own direction quoted verbatim

   Every agent reads the brief before touching anything.

---

3. Cast and build the bible (gate: casting test picked)

1. Casting test. Generate the same 3–4 scenes three ways, side by side: model A with refs, model B without refs, model B with refs. Pick one look for the whole video.
   • Creators' choice of image model:
     • GPT Image for instruction accuracy
     • Nano Banana for realism and real materials
     • Midjourney for stylized or cinematic looks, with a locked --sref code
   • On varg: gpt_image_2_5, nano_banana_2_1, soul (Higgsfield character styles). Outside varg: Midjourney.
2. Identity text. One paragraph describing the character's face and body, pasted word for word into every image and video prompt. Models read every word literally:
   • "long loose strands across her face" put hair over her eyes in 27 stills
   • the simile "like a microphone" put a real microphone in three of four pictures
   • So: no similes, no ambiguous hair or wardrobe phrases.
3. Reference sheets for anything that appears more than once: front, side, back, three-quarter and full body, on a blank background.
4. The bible. One page per recurring character, place and prop. Each page holds the identity text, the locked style, the sheets, and the prompts that worked.
   • Recurring rooms get a fixed description pasted into every prompt; furniture otherwise rearranges itself between shots.
   • Name screen positions explicitly ("Mia on the left, Jo on the right") when two characters share a frame.
   • After every session the agent writes a short note on what worked and updates the index.
   • Keep it as files in the project (bible/<name>.md + the sheet URLs); varg presets can hold reusable prompt settings.
5. Lock the references. Once locked, never regenerate them to fix motion. Fix the motion prompt instead; a new image undoes the consistency work.

---

4. Storyboard as data, on the grid (gate: storyboard approved)

storyboard.json, one entry per shot:

{ "id": "s034", "section": "chorus1", "start_bar": 41, "bars": 1,
  "start": 42.1, "end": 44.55, "kind": "singing",
  "lyric": "I keep on waiting for the light", "lyric_surface": "nutrition_label",
  "picture": "...", "graphics": "...", "camera": "slow push-in",
  "refs": ["lead_sheet", "diner_plate"], "resolution": "1080p" }

Rules:

• One shot per lyric line, plus the instrumental moments.

• Every cut sits on a downbeat: start and end come from songmap.downbeats, never from line starts. A singer rarely starts a line on the "one"; in one draft only 21 of 74 line-start cuts were on a downbeat.

• Three shot kinds:

  | Kind | Inputs | Mode | |---|---|---| | singing | approved still + a clean vocal slice containing only the words sung on camera | reference-to-video | | motion | still + the song itself, so the action lands on the beat; nobody sings | reference-to-video | | insert | silent; the still is the exact first frame | image-to-video (easiest to get right) |

• Spoken-dialogue scenes last a whole number of bars, so the music drops back in on the beat.

• Each clip is generated to start one beat before its shot, so a slightly late take still covers the cut.

• Choreography: pair one dance move with one cut,