Case study · Product Designer · Automattic
Turning long-form audio into short-form video, without a human in every loop.
Content Content is an AI-orchestrated platform that ingests a 60-minute podcast and produces five to eight vertical shorts, depending on what it finds, clipped, captioned, visually directed, and ready to publish. I designed the product, built the editor, and wired the pipeline that connects transcription, viral-moment detection, and generative visuals.
The problem
Podcasts are full of moments that would do well as short-form video, but they're buried in hours of audio, and the workflow to extract them is fragmented across transcription tools, NLEs, design software, and stock libraries.
Most existing platforms assume the source is already a video and the editor is already a professional. Audio-first creators are left manually scrubbing, cutting, captioning, and visually directing each clip. That kind of workflow does not scale well for a weekly show.
What I wanted to do was pull that whole chain into one place: find the moment, cut it, caption it, and give it a visual treatment without making the creator jump between five tools. If the default result was decent, the user could refine from there. If the system got it right often enough, the editor could fade into the background.
My role
Solo designer and builder. I owned the product end-to-end across the surfaces below.
- Product strategy
- UX architecture
- Interface design
- Frontend implementation
- Backend orchestration
- AI workflow integration
Design and implementation happened at the same time. I prototyped the UI directly in React so I could work against real pipeline output instead of static mocks. That also helped me figure out what the backend actually needed to return.
Product model
Three nouns. Every screen, route, URL, and table is built around them.
The dashboard shows workspace cards with a custom illustration so the view does not turn into a wall of text. That illustration was designed in Figma, then redrawn in code over a few focused AI-assisted threads because one-shotting a complex SVG usually creates more debugging than it saves. Opening a project drops the creator into a React Flow canvas of every viral clip the pipeline produced for that episode, with the editor panel docked alongside. Keyboard support mattered too. Spacebar plays, M mutes, and the shortcuts work across the app.
Constraints & tradeoffs
Automation vs. user control
Best case, a podcaster picks a template once and the pipeline keeps producing usable clips without much help. In practice, there still needs to be an editor for the moments when timing, captions, or visuals come out wrong. The same product had to support both hands-off and hands-on use.
Naming the wrong thing
The Supabase table that holds shorts is called `versions`. It made sense early; it makes no sense now. Renaming touches the whole pipeline, so it sits on a list of debts I track instead of hiding. That kind of naming honesty is part of being the only person on the project.
Three.js is the hard part
Most of the public material around Three.js leans toward game-engine examples. I was using it for typography, shader backgrounds, and browser-based motion work. I built a frame-rate overlay early because I cared more about how it felt in use than about abstract performance numbers.
Micro-seconds of doubt
During upload there's a brief stall before the custom animation kicks in. It's a half second, and it still reads as broken. I treat perceived latency as a design problem and queue fixes for it the same way I queue functional bugs.
AI output is non-deterministic, so it has to be reversible
The LeMUR clip picks and Gemini images were often good, but they were never consistent enough to trust blindly. I treated every result as something the user might need to adjust. Bad clip boundaries could be dragged. Bad images could be regenerated or swapped without rerunning the whole pipeline.
Render cost
Bespoke per-clip video generation is expensive and slow. The system composes cheap primitives in-browser (Three.js shaders, Troika text, cropped stock images) and only sends the final composition out to a GPU host for the actual render.
Design decisions
Three nouns: workspace → project → short
Creators think in shows and episodes, not in loose files. One workspace holds a show, one project holds an episode, and each project holds its shorts. Source audio, transcripts, clip data, and generated images all live under the project that owns them.
Default to the video tab first
Early on the project page defaulted to an info tab and I kept asking why I was always one click away from the thing I came to do. People are here to edit a clip; that's what loads first.
Built the UI from scratch
I built the interface from scratch instead of pulling from a component library. The depth, the overlapping panels, the hover treatment, and the motion timing all shape how the product feels. A stock UI would have flattened too much of that.
Thick resize handles, capped rotation, on-canvas guides
Small UX calls that came from using the editor on real clips: handles you can grab without aiming, rotation that stops before captions become unreadable, and a toggle that overlays TikTok/Reels/Shorts safe-zones so text doesn't collide with platform UI.
Icon row with hover labels (the tabs-within-tabs fix)
The editor needed three nested tab rows and ran out of horizontal space. Collapsing the middle row to icons with hover labels gave the hierarchy back without losing affordance.
Local editor wired to production Supabase
While iterating I had the local app talking to the live database, so any edit I made in dev was instantly visible in the deployed build. It made multi-surface debugging feel like one product instead of two environments.
Process & iteration
Agentic engineering
I built this the same way I usually build complex AI products: broad strokes first, then tighter and tighter passes as the shape becomes real. I started with a high-level spec for the application and used that to get the base architecture in place. Once the core surfaces existed, each deeper layer of functionality got its own smaller spec. That kept the work moving without pretending the whole product could be one-shotted up front.
That mattered here because the product changed a lot while I was making it. The first UX did not have a canvas. The editing model was different. The early versions also did not include generative AI or the more advanced caption system built with MSDF text. Those pieces showed up later as the workflow became clearer and the constraints became more concrete.
The process feels closer to painting than to assembly. You block in the composition, then add definition over time. Each pass makes the next decision easier. With an app like this, the UI, the prompts, the backend shape, and the user flow all keep informing each other. Even with good specs, I do not think sophisticated products land in a final state on the first try. If people are actually using the product, parts of it will always get expanded, simplified, cut, or rethought. That ongoing iteration is the work.
AI-assisted workflow
AI does a lot of the work here, but I kept each model in a narrow role. The outputs stay visible in the UI so they can be replaced, edited, or ignored when needed.
One upload, many reversible stages
The same thinking shaped the build process. Working in code with AI help made it faster to test ideas against real data instead of imagining them in mocks. The Three.js pieces still needed the most manual work. Shader and MSDF typography examples on the public internet tend to skew toward game-engine use cases, so those parts took more iteration.
What shipped, what didn't
This is what shipped and what did not. There were a lot of things I wanted to add, but some of them were simply too big to get done in the amount of time I spent on the project.
In
- Workspace / project / short hierarchy and dashboard
- Custom upload + processing animation
- Project canvas (React Flow) with viral-clip cards
- Editor: captions, typography, alignment, rotation, guides, transitions
- Three.js background shaders and MSDF caption effects
- AI Director: per-segment image generation with style templates
- Post-processing mixer (dither, layer toggles)
- GPU render pipeline
- Auth, billing scaffolding, account settings
Out (known)
- Template browser and template authoring
- Auto-scheduling and cross-platform publishing
- Cross-platform analytics layer and performance intelligence
- Video-generation clips that extend the AI Director from still images into full motion sequences
- Local-folder ingestion daemon
- Highlight-style sub-tabs (scale / underline / background)
- Workspace card reorder bug
- Dither scaling at large frame sizes
The biggest omitted product layer is analytics. I wanted a system that pulled performance data for each short across platforms, normalized it, and surfaced non-obvious patterns instead of just raw views and watch-time charts. The more interesting opportunity was an internal AI agent that could read that performance history, connect it back to caption treatment, pacing, visual style, topic, and hook structure, then suggest how the next batch should change.
Over time that could turn into a real feedback loop. Instead of simply telling a creator that a video performed well, the system could show which openings worked on Reels, which caption treatments held attention on Shorts, and which visual styles were weak on TikTok. That is where the product could start helping users improve over time and eventually generate platform-specific styles by default.
The product
Five short walkthroughs of the dashboard, the editor, the AI Director, the typography and shader system, and the orchestration pipeline.
Reflection & learnings
This project made one thing very clear to me. The hard part is rarely the prompt. The hard part is the surface around the model, especially the part that helps the user step in when the output is close, but wrong.
What changed for me over the last year and a half was AI. Once the models got good enough and the agentic harnesses got better, I could build full-stack products on my own in a way that did not feel realistic before. That makes the work feel less like traditional product design and more like product engineering. I still care about strategy, UX, interface, and system behavior, but now I can carry those decisions much further into the actual implementation.
I also ended up with a pretty sober view of the product's lifespan. In the short term it would be useful, especially for non-technical creators who want an end-to-end system instead of stitching together transcription, editing, prompting, and rendering tools themselves. But as agentic AI systems become normal and more people get comfortable directing general-purpose tools such as Claude Cowork, the defensibility of a narrowly scoped wrapper like this collapses fast.
My honest belief was that the moat was close to zero and the window was probably one to two years at best. The workflow insights were real, but the product advantage was too easy for broader agentic systems to absorb. That was the main reason I abandoned it. I did not stop because the problem was fake. I stopped because I did not think the business would stay durable for long enough to justify continuing.