video-extract-mcp

Give an AI agent any video link: download the file, read the transcript, or get just the frames that matter — all on your own machine.

Two jobs, and you can use either on its own:

Get the video. A YouTube link, a TikTok, a WeChat Channels share URL, a raw .mp4, or a page from a site nobody has heard of — it resolves and downloads it, whole or just the section you asked for. If that is all you need, stop there; nothing forces you to analyse anything.

Or read it. A transcript (real captions when the platform has them, local speech recognition when it does not) and a small set of important keyframes — deduplicated, scene-aware, and scored — instead of a thousand near-identical stills.

Built for AI agents. Three MCP tools, no cloud, no API keys, no Python.

---

What it can do

Get the video, from almost anywhere. Paste a link and it resolves: YouTube, TikTok, Instagram, X, Facebook and Reels, Twitch, Vimeo, Reddit, WeChat Channels, a bare .mp4 or .m3u8, or a page on a site nobody has heard of. Local files work too. Anything unsupported comes back as a clear failure rather than a crash.

From there you choose how much work to pay for:

• Just look it up. Title, creator, duration, chapter list, description, comment count — without downloading a byte. On a long video this is how you find the one section worth analysing.
• Download the file. The whole thing, or just seconds 300–420 of it. Where the platform supports ranged fetching, only that section is transferred rather than the full video.
• Get a transcript. Real captions when the platform has them, in any language it publishes; local speech recognition when it has none. The result says which you got. A captions-only request skips the download entirely — seconds instead of minutes.
• Get the frames that matter. Not every Nth frame: scene changes, on-screen text appearing, genuinely new visuals — deduplicated and scored, typically a few dozen for an hour of video. Or uniform sampling, or one exact frame at one exact second, if that is what you need.
• Do several at once. Pass a list of videos and get a result per video; one failing does not sink the rest.
• Reach private or rate-limited content. Point it at your browser's cookies and it can fetch age-restricted, members-only or followers-only media, and shrug off the bot checks that block anonymous downloads.
• Run long jobs in the background. A full analysis of an hour-long video takes minutes; it can run as a background task, report progress over a local HTTP endpoint, and hand back the result even if your client gave up waiting.

Everything happens on your machine. No API key, no upload, no third-party service.

Why this exists

Two problems, really. Getting the video at all — every platform hides its media behind a different mechanism, and none of them want a script fetching it. And then reading it: an LLM cannot watch a video, and the usual workaround — dump every Nth frame into the context window — burns enormous amounts of context on frames that are 98% identical to the one before, while still missing the slide that changed when nothing else moved.

video-extract-mcp handles both:

• Fetching, from almost anywhere. One resolver chain covers the big platforms, direct media URLs, and generic sites, with ranged fetches where the platform allows them and cookie support for anything that needs a login. Downloading is a complete use of this tool, not a step on the way to something else.

• Transcript, honestly sourced. The platform's own captions are used whenever the video has any — human-written first, otherwise the platform's automatic ones. Audio is transcribed locally (Whisper or SenseVoice) only for videos with no captions at all. The result tells you which you got, via transcript.source.

• Keyframes chosen, not sampled. Scene-boundary detection, blur/quality filtering, on-screen-text novelty (subtitle-aware, so burned-in captions don't preserve redundant frames), and image-embedding similarity feed an iterative diversity-aware selector.

• Output goes to disk, not into your context. The tool reply is a compact summary plus file paths. A 35-frame manifest and a full transcript don't belong in a conversation where the agent needs three numbers from them.

• Everything runs on your machine. No third-party API, no upload, no key. Long analyses can run as MCP background tasks — the tool returns a handle immediately and pushes progress; see Background tasks below.

Quick start

Install the system binaries first — these can't come from npm:

# macOS; use your package manager elsewhere
brew install ffmpeg yt-dlp tesseract tesseract-lang

Then point your MCP client at the package. There are two ways, and they differ in ways worth thirty seconds of your time.

Option A — npx, nothing installed. Simplest, and it picks up new releases on its own.

Claude Code:

claude mcp add --scope user video-extract -- npx -y @yanlinglabs/video-extract-mcp@latest

Codex:

codex mcp add video-extract -- npx -y @yanlinglabs/video-extract-mcp@latest

Another agent? Point it at SKILL.md and it can install itself.

Keep the @latest — without it npx pins to the first version it cached and never updates.

Option B — installed globally. Starts faster and gives you the video-extract status CLI as a real command.

npm install -g @yanlinglabs/video-extract-mcp

Then register it — Claude Code:

claude mcp add --scope user video-extract -- video-extract-mcp

Codex:

codex mcp add video-extract -- video-extract-mcp

| | npx (A) | global install (B) | |---|---|---| | Updates | automatic only with @latest in the spec — a bare npx -y @yanlinglabs/video-extract-mcp pins to the first version it cached and never updates | manual: npm update -g @yanlinglabs/video-extract-mcp. You stay on the installed version until you run it | | Startup | ~0.9s (npm resolution on every launch) | ~0.1s | | video-extract status in your shell | not on PATH — needs npx -y -p @yanlinglabs/video-extract-mcp video-extract status | works directly | | Working directory | must not be this package's own checkout (see below) | irrelevant |

Neither affects what agents can do: an agent checks on background work over HTTP using the statusUrl handed to it in the reply, never a shell command. The CLI is for humans.

Or in any MCP client's config — "command": "npx", "args": ["-y", "@yanlinglabs/video-extract-mcp@latest"] for A, or "command": "video-extract-mcp" with no args for B:

{
  "mcpServers": {
    "video-extract": {
      "command": "npx",
      "args": ["-y", "@yanlinglabs/video-extract-mcp@latest"]
    }
  }
}

One gotcha with npx, and it only bites contributors. Run inside this package's own git checkout, npx @yanlinglabs/video-extract-mcp fails with command not found — npx sees the local package.json claiming that name, looks for the binary in a local node_modules/.bin that was never populated, and gives up. Since MCP clients launch servers with the working directory set to your project, option A cannot work in this repo. Working on the tool itself? Point that one project at your build — claude mcp add --scope local video-extract -- node "$PWD/dist/mcp.js" — which also means a npm run build takes effect immediately, with no publish round-trip. Everywhere else, npx is fine.

That is enough for any video that has captions — which, thanks to the caption-first transcript policy, is most of them. The vision model downloads itself on first use.

Speech models are only needed for videos with no captions at all, and they are fetched automatically the first time one is. Only the engine that video needs is downloaded — 233 MB for the Chinese/Japanese/Korean model, 1.3 GB for Whisper — into ~/.cache/video-extract-mcp/models. Set VIDEO_EXTRACT_AUTO_FETCH_MODELS=0 to keep it manual, or pre-fetch them yourself:

npx -y @yanlinglabs/video-extract-mcp --help   # installs the package
curl -fsSL https://raw.githubusercontent.com/yanlingLabs/video-extract-mcp/main/scripts/fetch-models.sh \
  | bash -s -- ~/.cache/video-extract-mcp/models

~/.cache/video-extract-mcp/models is where the tool looks by default. Override with VIDEO_EXTRACT_MODELS_DIR. If the fetch is disabled or fails, an uncaptioned video still returns frames and records a warning explaining why the transcript is missing — it degrades rather than fails.

From source (contributors)

git clone https://github.com/yanlingLabs/video-extract-mcp.git
cd video-extract-mcp
npm install && npm run build
./scripts/fetch-models.sh    # into ./models, which takes precedence when present
npm run preflight            # verifies ffmpeg / ffprobe / yt-dlp / tesseract

Environment variables

| Variable | Purpose | |---|---| | VIDEO_EXTRACT_MODELS_DIR | Where speech models live. Defaults to ./models when that exists, else ~/.cache/video-extract-mcp/models. | | VIDEO_EXTRACT_AUTO_FETCH_MODELS | Set 0 to stop the speech models being downloaded on demand. They are then your job (scripts/fetch-models.sh), and an uncaptioned video degrades with a warning saying so. | | VIDEO_EXTRACT_COOKIES_FILE | Path to a Netscape-format cookie jar, used for every yt-dlp source at once — YouTube, Instagram, Facebook, X, TikTok, Twitch and the rest. See Authenticated sources. | | VIDEO_EXTRACT_COOKIES_FROM_BROWSER | Load cookies from a local browser instead: chrome, firefox, safari, edge, brave, chromium, opera, vivaldi, whale, optionally browser:profile. Ignored when VIDEO_EXTRACT_COOKIES_FILE is set. | | VIDEO_EXTRACT_WECHAT_COOKIE | A yuanbao session cookie, required only for WeChat Channels links. Separate from the above by design — a different protocol with its own credential. | | VIDEO_EXTRACT_MAX_CONCURRENCY | Caps concurrent analyze_video item executions — plain calls and background tasks, batch items and separate calls, all count against the same limit. Default 4. resolve_video is exempt: it loads no models, so there is nothing to throttle. | | VIDEO_EXTRACT_TASK_TTL_MS | How long a completed background-task handle stays queryable before it expires. Default 1800000 (30 minutes). 0 (or any non-positive value) means the handle never expires. Governs the in-memory handle only — files already written to destinationPath are never deleted by the tool, expired handle or not. | | VIDEO_EXTRACT_STATUS_PORT | Pins the port of the localhost /status endpoint (see Watching progress). Unset picks an ephemeral port each start (default: endpoint on). The literal value 0 disables the endpoint entirely — note the contrast with VIDEO_EXTRACT_TASK_TTL_MS above, where 0 means no expiry, not disabled. |

Authenticated sources

Plenty of media is not public, and the answer is the same one the platforms themselves ask for: cookies. One jar covers every yt-dlp source at once — cookies