AI-Powered UGC Video Production Engine

Ongoing commercial project. The source code is private.

The product

An engine that turns a product page URL into a publish-ready vertical UGC video: 20 to 35 seconds, AI voice-over, animated visuals, on-brand overlays, hashtags and pinned first comment, all bound to one of five daily editorial slots. Operated solo, with human input reduced to the few points where judgment matters: five minutes of review per video, five minutes of analytics a day. The engine does everything else, at an average cost of about $3.5 per video with a hard budget ceiling enforced in code. Every video is treated as a measurable experiment rather than a handcrafted production.

Architecture: judgment vs determinism

The core design decision: the LLM intervenes only where judgment is irreplaceable, everything else is deterministic Python.

• Five LLM boundaries, each justified by a documented failure: structured product extraction, script writing, visual scenario, a vision fidelity gate, and publishing-slot scoring. Slot arbitration is the cleanest expression of the pattern: the LLM scores every product-slot pair, Python computes the optimal assignment, a human applies it.
• Structured outputs enforced by validators, not by trust: agents write YAML fragments that a deterministic merge CLI validates, defaults and canonicalizes. The exit code is the contract.
• Prompts as a runtime-editable surface: hot-reloaded prompt templates and agent definitions, per-step model routing (lightweight models for glue steps, stronger ones for creative writing and vision), and an audit trail recording which prompt revision produced which section of which video.
• One multimodal gateway: image generation (GPT Image, Nano Banana), image-to-video (Kling, Seedance), ElevenLabs for voice, Claude for orchestration, unified behind a single client with one ledger and one credit currency.

Production pipeline

• Structured extraction: product record with key features, USP, hook angles and a strictly verified reference packshot.
• Script per editorial angle: five daily slots (educational SEO, social proof, mechanism, demo and routine, conversion), a library of narrative structures, and anti-repetition rules checked against the account’s recent videos. The script step always pauses for human review: it is the one step where a correction changes everything downstream.
• Audio-first production: text-to-speech then silence-based slicing; the real voice track drives block durations and overlay timings.
• AI frames with a fidelity gate: every product-bearing frame is compared by a vision LLM against the reference packshot, catching brand and label hallucinations before money is spent on animation. Rejected frames are regenerated with failure-augmented prompts, twice at most, then escalated to the human.
• FFmpeg assembly: 1080x1920 master, overlays rendered as PNGs and placed by saliency detection inside the platform’s safe zones, brand accent colors extracted programmatically from the packshot, loudness normalization to broadcast standards.

Reliability and cost observability

• Pre-spend guards: budget estimation aborts a run before any paid call if it exceeds the ceiling; provider quota is verified fail-fast.
• Three-layer cost tracking: a submit-time ledger so a killed session never loses a paid task, orphan reconciliation that recovers already-billed results, and balance checks against the provider account to catch any call escaping the ledger.
• Content-hash idempotence: frame and animation inputs are fingerprinted separately, so editing an animation prompt regenerates the clip only and never re-pays for a validated frame.
• Truthful-outcome verifiers: post-run hooks re-read the pipeline state and downgrade to failure any run that exited green without actually advancing, a recurring failure mode of LLM-driven automation.
• A suite of 122 automated tests covers the pipeline, the analytics modules and the API.

Analytics loop

• Daily ingestion of distribution and commerce metrics: per-second retention curves, views, shares, product clicks, orders. Production cost is converted at historical exchange rates so per-video ROI stays honest across months.
• Automated daily briefs: amplification alerts on velocity thresholds, retention gates, funnel diagnosis pointing at the weak link.
• Weekly reviews with a frozen-cohort protocol: each A/B cohort runs under a frozen creative regime, five replicates minimum, and learnings land as a block on the next cohort. Iteration stays in the cheap zone, at the script stage, never in paid re-production.
• A measured negative result: automatically feeding top performers back into the generation prompts caused stylistic mode collapse. The loop is now closed through human-reviewed editorial rules, versioned in the prompt layer.

This measurement discipline drove the project’s main pivot: moving from avatar lip-sync videos to a fully faceless art direction after the data showed an order-of-magnitude gap in median views, and a further 20 percent cost reduction per video through a controlled model A/B.

Operator console

The operator console covers the whole lifecycle: publishing queue with slot planning, per-video editor, prompt management and analytics dashboards.

Approach

Organic content serves as the R&D lab: every published video feeds the measurement loop, converting formats are identified then amplified. A complete product approach, from distribution-algorithm analysis and compliance guardrails encoded in prompts to conversion tracking, executed with the rigor of an engineering pipeline.