Open source · video-factory

Edit video by talking to Claude, in any language

Give it raw footage or a script and say what you want: cut the interview by topic, a lower third for the CEO, a chart, captions, a vertical version. A Claude Code skill makes the editing decisions and a command-line tool does the work. You get back a finished video, and editing runs on your machine.

Install Source on GitHub

MIT licence
macOS or Linux
Any language Whisper speaks
Right to left built in

In the browser

The whole edit on a timeline, in one HTML file

Every render can come with a review page: the video, and under it every track the way an editing app shows it, parts and transitions, graphics, b-roll, captions and music, with a playhead that follows playback. It needs no server and no editing app. Drag a part to reorder it, click a transition to change it, pause to leave a note; it all comes back as text Claude applies.

What it does

From raw footage to a finished video

A time for every word

Transcription runs locally with whisper.cpp. The project remembers its language, detected or set, and every later step follows it. Hebrew uses the ivrit.ai model; every other language the multilingual one.

Cuts by meaning, with your approval

Claude splits the transcript into parts of one idea each and proposes a story, with a reason for every part. Nothing renders until you approve it. Hesitation sounds and long pauses come out, and captions follow the cuts.

Captions and graphics that read the right way

Burned-in captions with karaoke, and titles, lower thirds, counting numbers, charts and end cards from one brand file. In Hebrew or Arabic they lay out right to left.

Voice-over from a script

A narrated script sets the timing of a product video. ElevenLabs, Azure, Google and OpenAI voices, or the free macOS voice to time the edit first. Captions follow the narration word by word.

Versions and shorts

Vertical, 30-second and 15-second versions from one story. From a long video, candidate moments are ranked from the transcript, Claude picks, and each clip renders vertical with a hook title.

Client notes and editor hand-off

A review page in one HTML file, with no server: every track on a timeline, drag to reorder, timestamped notes that come back as text. The edit also exports to Final Cut, Resolve or Premiere.

What you can ask

Say it in your own words

These are examples, not fixed commands. Claude asks only when the answer changes the result.

“Make a 45-second promo for ACME from ~/Footage/acme: the CEO interview, factory b-roll, our logo and colours, a chart of customers by year, music, and a Reels version.”
“Write a 30-second script for our app, narrate it with Azure, and cut the screen recordings in ~/Desktop/app to it.”
“Find the best moments for Reels in this podcast: five clips of 30 to 45 seconds.”
“Transcribe the lecture, take out the ums, and add karaoke captions.” In Hebrew, right to left.
“Make a review page for the client.” Their notes come back as text Claude applies.
“There’s no talking, it’s travel footage. What’s in it?” Claude reads timecoded contact sheets; a vision model you choose can watch too, on your machine by default.

Install

FFmpeg, whisper.cpp and a model

You need macOS or Linux with Python 3.10+, FFmpeg with libass, whisper.cpp with a model, Node.js 22+ and Chrome for the graphics, and Claude Code.

FFmpeg with captions, and whisper.cpp (macOS)

brew install ffmpeg-full

A transcription model, every language

pip install -U huggingface_hub && hf download ggerganov/whisper.cpp ggml-large-v3-turbo.bin

Get it, check it, start Claude

git clone https://github.com/yybd/video-factory && cd video-factory && python3 tools/vf.py doctor && claude

For Hebrew, add the ivrit.ai model: hf download ivrit-ai/whisper-large-v3-turbo-ggml ggml-model.bin. It is far better at Hebrew, and answers in Hebrew whatever is spoken, so other languages use the multilingual model. vf.py doctor says what is missing. API keys are needed only for a paid voice or a cloud vision model, and go in .env, which git ignores.

Good to know

Where it stops

There is no editing window. You work in a conversation. The review page and the timeline export are how you see the edit and hand it on.
No generated footage, avatars or lip-sync, and no stock library.
Shorts are picked by judgement, Claude’s, plus an optional visual score, not a trained virality model.
What leaves your machine: the script text, when you choose a paid voice; footage, only if you choose and allow a cloud vision model.
It is new. It was tested end to end on test clips with synthetic voices, in Hebrew, English and Spanish.

Writing

The article about video-factory

Edit video by talking to Claude: in any language, right to left included
Six uses, from a promo cut out of an interview to shorts from a podcast and a Hebrew lecture with right-to-left captions, and where it stops. November 2026.

he

More open source

From the same desk

App Factory
Claude Code skills

43 skills that build, describe, ship and announce apps, to the App Store and Google Play and outside them.
Project page
RepoGuards
Claude Code hooks

Refuses a blind commit or a push into a repo the session is not standing in, and shows the agent what is there.
Project page
telegram-notify MCP
MCP server

A send-only MCP server that lets Claude message you on Telegram, to one chat fixed in configuration.
Project page