Edit video with an AI assistant

In September I spent nine days in Chicago with a DJI Osmo and two iPhones, 40 to 70 clips a day. Each day became a 2 to 5 minute trailer and a 10 to 20 minute episode, and at the end a 55-minute film of the whole trip, cut by theme. The assistant did the editing from my words. I watched, judged and asked for changes.

It was not one click. The first cut is almost always boring or out of sync; the third one is good. That is still far faster than editing by hand, and the boring part (watching every clip, finding the beat, syncing subtitles) is gone.

The tool

Life2Film Studio, a free desktop app for Apple Silicon Macs (early beta). Everything runs on your Mac; nothing is uploaded.

brew install --cask fortunto2/tap/life2film-studio

(No Homebrew? Download the .dmg from the release page.)

It analyses the footage (shots, picture quality, bad frames), transcribes speech on the Mac’s GPU in any language, builds a montage cut on the beat, renders an mp4, and publishes to YouTube, private first.

Connect Claude Code to the open window:

claude mcp add --transport http life2film http://127.0.0.1:8421/mcp

Then ask in plain words:

Add the folder ~/Movies/trip/day1 to Life2Film and wait for the analysis. Take the first track from ~/Music/score. Build a 3-minute montage with the Travel profile, render it, and show me a contact sheet.

How a day becomes a film

1. One folder per day, every camera in it. Take the shooting time from the file’s metadata, not its name: my DJI names were an hour off and the cameras drifted apart.
2. Analyse picture and speech. About 20 minutes for 66 clips the first time, seconds after that.
3. Read the day as text. All the speech of the day in one file. Every film starts there: the assistant reads it like a scriptwriter and proposes the story.
4. Cut the trailer: voice as one continuous track, pictures on top. The episode is the same with a bigger budget and whole conversations kept.
5. Look at the whole film as one picture. A contact sheet of the full film shows a sunset among night shots, or three talking heads in a row, faster than watching it.
6. Subtitles in two languages. On YouTube as tracks; on Reels burned into the picture, because people scroll with the sound off.
7. Publish private, watch on the phone, then make it public.

The same way, a week in California became a trailer:

Rules I learned on bad cuts

• The cutaway shows what is being talked about. Talking about the Bean, the Bean is on screen.
• Never cover a shot where lips move. Viewers read it as broken sync.
• Open cold: 8 to 10 seconds of the best shots, then the hook. Pick those shots yourself. The machine picks sharp and colourful, not interesting.
• Live music stays one piece. Cut it per shot and the beat restarts at every cut.
• Music playing in the background is a copyright claim. Car radio, a shop, a festival stage: one day got 10 claims. Strip it and put your own score under it.
• Vertical clips go to Reels as they are. Cropping wide shots blindly to 9:16 kept 5 faces of 37; following the faces kept 31.
• Teach it your names. Speech recognition turns unknown names into celebrities. Keep a word list for the trip and fix it once.

The technical side is in project-life2film. For the soundtrack, see creative-music-with-claude.

Behind the app

For my own trip I mostly skipped the window and ran the same core with ffmpeg from the terminal. Four posts on what that month taught: creative-video-transcribe (speech as the timeline, two languages, the GPU path), creative-video-agent-cut (the cut as a script, the critic, the trip film), creative-video-sound (your own score, stripping the radio) and creative-video-shipping (codecs, vertical crops, the hook and the cover). The rules are published as two skills in solo-factory: solo:transcribe and solo:video-edit. All the skills, video and music, are listed in creative-skills.