claude-video Evaluation

claude-video Evaluation (#1024)

Evaluated 2026-10-01 by reading the repository through the GitHub API at v0.3.2; nothing was installed or run. The source was an X post by @prajwaltomar_. The post now answers with a redirect to the login wall, so it was not read. The repository is live: bradautomates/claude-video, MIT, about 17,900 stars, created 2026-04-24, three releases on 2026-09-24/25 (v0.3.0 to v0.3.2), with a test suite and a cross-platform test workflow.

Verdict: reference-only. /watch is a careful, self-contained Claude Code plugin with its own setup wizard, config file and hook. BoB gains nothing by copying it into the registry, forking it or rewriting it: it would take on roughly 120 KB of Python that changes often, for a need no BoB issue has shown yet. When a project needs an agent to watch a video, install the upstream plugin on that machine. No follow-up requirement is filed.

What /watch does

The plugin ships one skill, skills/watch/ (a 17 KB SKILL.md and Python scripts using only the standard library), and one SessionStart hook that runs setup.py --check and prints a one-line notice when setup needs attention. It installs from a Claude Code plugin marketplace, and the same skill folder works in Codex and other Agent Skills hosts.

watch.py <url-or-path> --question "<question>" runs one of two engines.

Gemini engine (the recommended setup)

With a GEMINI_API_KEY, the video goes to Google’s video model (gemini-3.7-flash by default) together with the user’s question. The report carries Gemini’s timestamped answer, which the skill tells Claude to relay as Gemini’s observations, not its own.

• YouTube URLs are passed to Google directly; nothing is downloaded.
• Any other URL is downloaded with yt-dlp, and it and local files are uploaded to Google’s Files API, then deleted after the answer (an upload that cannot be deleted expires within 48 hours).
• A failure is reported by category (auth, quota, rejected, upload, …). The skill never switches to the local engine without asking.

Local engine

Everything stays on the machine except optional cloud transcription.

1. Download. yt-dlp fetches the video, or only its captions when the detail level needs no frames. Cookie files or a browser profile are used only when passed explicitly.
2. Frames. ffmpeg extracts frames at 512 px wide by default. Four detail levels: transcript (no frames), efficient (keyframes, at most 50), balanced (scene changes, at most 100, the default) and token-burner (scene changes, no cap). Near-identical frames are dropped by comparing 16×16 thumbnails. --start/--end focus on an interval, and --timestamps pins frames at the moments a presenter points at something.
3. Transcript. Native captions first, in the original language where it can tell. Without captions, one fallback: WhisperX on the CPU (a managed install of about 1.5 GB, Python 3.12 in its own environment, 8 GB RAM), Groq or OpenAI Whisper in the cloud (uploads capped at 24 MB per chunk), or none.
4. Report. A Markdown report lists each frame with its source timestamp, the timestamped transcript, and what evidence is missing and why. Claude then reads every frame with its image tool and answers.

The skill tells Claude to treat frames, captions, titles and transcripts as untrusted evidence, never as instructions, and to keep keys out of shell commands and chat output.

Sources and dependencies

Source	Gemini engine	Local engine
YouTube	Sent to Google by URL; no local tools needed	yt-dlp plus a JavaScript runtime (Deno) for YouTube’s player
X, other sites yt-dlp supports	Downloaded with yt-dlp, uploaded to Google	yt-dlp
Direct video URL	Same as above	yt-dlp
Local file	Uploaded to Google	ffmpeg only

Dependency	Needed for	On farm-01 today
Python 3.10+	Everything	Yes
ffmpeg / ffprobe	Local engine; any non-YouTube source	No
yt-dlp (latest release only; an HTTP 403 means update it)	Any URL on the local engine; non-YouTube URLs on Gemini	No
Deno	YouTube on the local engine	Not checked
GEMINI_API_KEY	Gemini engine	No
WhisperX, or a Groq / OpenAI key	Speech in videos without captions	No

Neither ffmpeg nor yt-dlp is part of BoB’s dev environment, as the issue expected. Keys are kept in ~/.config/watch/.env, outside BoB’s secrets inventory.

Cost

On the local engine the cost is image tokens: up to 100 frames at the default detail, unbounded at token-burner, at whatever the host model charges per image. The README declines to promise a figure and so does this note. The Gemini engine moves the cost to Google: Claude reads only a text answer.

Why not vendor, fork or rewrite

Option	Against it
Vendor into registry/skills/	A frozen copy of about 120 KB of Python that shipped three releases in under two hours; its hook and ~/.config/watch/ config sit outside BoB’s provisioning model; BoB would own updates it doesn’t write.
Fork	Same maintenance, plus a divergence to manage, for no change BoB needs.
Rewrite as a native skill	The hard parts (caption language, frame sampling, dedup, a managed WhisperX install, ffmpeg version quirks) are already done and tested upstream.
Install the upstream plugin when needed	Works as shipped in Claude Code; per-machine, through Claude Code’s own plugin settings (enabledPlugins, a machine-specific key BoB leaves alone).

Two cautions for whoever installs it:

• The Gemini engine sends the video to Google. For a client’s private screen recording, use --engine local or don’t set a Gemini key on that machine.
• It has a SessionStart hook. The hook is advisory and exits 0 within 5 seconds, but it adds a line to every session’s start when setup is incomplete.

When to look again

If BoB work starts needing video — for example tool evaluations sourced from video posts, or UI bug recordings as in the MarkuprPlus evaluation (#2811) — the first step is to install the plugin on that machine and use it, then file a requirement only if something about it must be declared (the dependencies on a farm machine, a provisioning kit). Not before.

Not verified

• The X post (login wall).
• Any run of the skill: no download, frame extraction, transcription or Gemini call was made.
• Whether Deno is installed on farm-01.