Timelines for Agents
a 9-product teardown

Swift NLE with frame-integer clips on typed tracks, per-property keyframe lanes, linked A/V groups, nested timelines, multicam, ripple-delete of ranges. The agent surface is the deepest studied: a local MCP server with ~44 filmmaker-intent tools where every mutation returns a delta (changed clips + compressed shift rules), undo is itself a tool, and inspect_timeline renders composited frames so the model can see its edit. Skills are SKILL.md playbooks; a meta-skill reverse-engineers a finished timeline into a reusable playbook.

An ECS engine (frames @30) under a canvas-rendered timeline with filmstrips, waveforms and keyframe rows. The most rigorous specs in the field: timing split into two questions (start/end placement vs sourceIn/sourceOut window, invariant enforced); scrub-then-settle seeking (paint the covering GOP keyframe instantly, exact-seek on rest); flash-free splits that hand the primed decoder to the right half; a fully-specified Premiere-style overwrite drop model. Agent contract is the self-describing dapi CLI — patches go through the same setters as the UI, same undo, plus perception commands (filmstrip, waveform, transcribe, capture).

The best transcript↔timeline fusion: word-level dual-engine timestamps; deleting words splits at word boundaries and ripples — with the sharpest scope rule studied: "only the track you're editing ripples; other tracks never shift." Silence is compressed, not deleted (>3s→200ms, >1s→600ms, user slider). Around it, a real NLE: V/N/B modes, JKL, Q/W trim-to-playhead, on-clip fade handles and volume lines, six keyboard preset schemes (Premiere/Resolve/FCP/CapCut), version snapshots that restore through the undo stack, FCP7 XML export. The agent presents a plan, then applies each step as an ordinary undoable edit while the timeline stays visible.

No timeline document exists — time structure is the React tree, and the Studio derives tracks from it at runtime. Its "read-only timeline" reputation is obsolete: clips drag, trim (left-edge trim writes from+duration+trimBefore together, rate-compensated), split, duplicate, reorder, keyframe — and every GUI edit is an AST codemod written into your source, so human drags and agent text-edits compose in one file with server-side undo. Editability is a static-analysis contract: non-analyzable code grays out with a reason. Ships 12 agent skills, editor plugins, a system prompt, and per-clip "Copy context for agents."

"Cursor for video editing": a real browser NLE (ripple toggle R, razor, snapping, keyframe lanes, beat-synced multi-clip edits, custom keybindings, Premiere XML + Resolve handoff with timecode preserved) driven by a "Director" chat with model picker. Three ideas worth copying: chat checkpoints that roll timeline state back to any message; interactive pickers mid-run (audition music before the agent proceeds); and agent vision self-review of rendered frames (v0.34). Frame-pinned comments on share links.

CoAnimator's architectural twin: clips are DOM elements with data-start/data-duration/data-track-index, rendered deterministically frame-by-frame — and its free local Studio has a real timeline: drag-retime, cross-track drag, edge trim, razor, snap toggle, keyframe diamonds with bezier easing, pointer gesture recording (R), music beat-analysis markers, word-level caption lane with a sidecar overrides file. Agent story is identical to yours: agent and Studio edit the same files, and every Studio dead-end emits an "Ask Agent" prompt pre-filled with project context.

A node-graph clipping factory (iHeartMedia at 150M views) whose editor is deliberately thin — but whose public API leaks the cleanest edit representation studied: a clip is keep-ranges over one source (segments: [{startSeconds, endSeconds}], validated non-overlapping) plus word objects {index, start, end, speaker, isDeleted} patchable by text, time-range, or index. Cuts snap to sentence boundaries with ±5s slack kept for later trimming. Three verbs per transcript selection — Cut / Mute / Edit-caption — and an MCP server mirroring the whole human surface.

By Revideo's own creator — and he made the move that matters: migrated from TypeScript scenes to a declarative XML format specifically because agents co-edit documents more reliably than imperative code. Tween/scene timeline (labels show from→to values), keyframes + easing, cloud render queue, an embeddable live scene runtime, and MCP + CLI as the product — MCP tool calls are the metered billing unit (100/week free, 1M/week on $20 Pro).

The anti-timeline pole, done well enough to matter: video → scenes → live editable elements, "no regenerating entire scenes, no hallucinated artifacts," Cursor-style chat checkpoints. But time itself is untouchable — API duration is a bucket (<10s…1-5min), the only API edit primitive is a whole-video followup, and no scrub/trim/retime surface exists anywhere. PH users still called element editing "the unlock" — proof of demand for regeneration-free editing, and proof nobody in this cluster can nudge a scene 500 ms.