Automated Video Production Pipeline

Firecrawl-Powered Content Extraction

The ingestion layer uses Firecrawl to extract structured content from any URL, including JavaScript-rendered single-page applications, paywalled articles with accessible previews, and multi-page long-form content. Firecrawl returns clean markdown with metadata including title, author, publication date, and extracted images. The system parses this output into a normalized dataset format that captures the article's hierarchical structure: main thesis, supporting arguments, key data points, and quotable passages. This structured representation becomes the input contract for downstream script generation.

For batch operations, the crawler accepts sitemap URLs or RSS feeds and processes entire content archives into indexed datasets. Each extracted article is deduplicated against previously processed content using perceptual hashing of the normalized text, preventing redundant video production from syndicated or republished articles. The crawler respects robots.txt directives and implements polite crawl delays to maintain good citizenship with source domains.

LLM Script Transformation

The script generator ingests the structured dataset and produces a complete video script divided into timed scenes. Each scene includes narration text with embedded SSML-compatible annotations for emphasis, pauses, and pronunciation guidance. Alongside the narration, the generator outputs a storyboard prompt for each scene: a detailed image description specifying composition, style, color palette, and mood that the visual generation stage will use to create matching imagery. The LLM also produces a video-level metadata package containing an optimized title, description, tags, and three candidate thumbnail concepts. Scene transitions are annotated with suggested motion types (cut, dissolve, wipe) based on the narrative pacing and emotional arc of the script.

Visual Generation Pipeline

Each storyboard prompt feeds into fal.ai's inference infrastructure running Stable Diffusion XL and Flux models. The system generates multiple candidate images per scene at 1920x1080 resolution, then scores each candidate against the storyboard prompt using CLIP-based similarity metrics. The highest-scoring image advances to the assembly stage. Style consistency is maintained across scenes by injecting a persistent style directive into every prompt: aspect ratio, color temperature, artistic style (photorealistic, illustration, cinematic), and subject framing rules. This ensures visual coherence across a 10-minute video even though each frame is generated independently.

For scenes that benefit from motion, the pipeline routes the selected still image through the Wan video generation model, which produces 3-to-5-second animated clips with natural camera movement and subtle environmental motion. The Wan model excels at parallax effects, slow zoom, and atmospheric animations that transform static compositions into cinematic sequences. Runway's gen-3 model serves as an alternative provider for motion generation, activated automatically when the Wan model queue exceeds latency thresholds or when the scene's motion requirements exceed Wan's capabilities (complex character animation, fluid dynamics, etc.).

Neural Voice Synthesis

ElevenLabs' Turbo v2.5 model synthesizes the narration audio from the annotated script. The system maintains a library of voice profiles tuned to content verticals: an authoritative baritone for business and finance, a warm conversational tone for lifestyle and education, and an energetic delivery for technology and entertainment. Voice cloning capabilities allow channel operators to establish a consistent narrator identity that listeners associate with their brand. The narration module segments the script into timed blocks aligned with scene boundaries, generating individual audio files with precise start and end timestamps. The assembly engine uses these timestamps to synchronize visual transitions with speech cadence, ensuring that scene changes land on natural sentence breaks rather than interrupting the narrator mid-thought.

Node.js Orchestration Engine

The entrypoint module orchestrate.js implements a directed acyclic graph (DAG) of pipeline stages. Each stage declares its input contract and output schema, enabling the orchestrator to validate data flow between stages at runtime and fail fast on schema violations before expensive API calls are made. The DAG topology supports conditional branching: if a scene's storyboard prompt includes motion keywords, the orchestrator routes through the animation generation path; otherwise, it skips directly to still-image compositing, saving both time and API costs.

Retry logic uses exponential backoff with jitter to prevent thundering-herd effects when a provider recovers from an outage. Each provider integration includes a circuit breaker that opens after three consecutive failures, redirecting traffic to alternative providers for a configurable cooldown period before attempting to re-establish the primary connection. This architecture ensures that a fal.ai rate limit at 2 AM does not leave a batch of 50 videos stalled until morning.

Cost Tracking and Unit Economics

Every API invocation is metered and attributed to the specific video that triggered it. The orchestrator maintains a running cost ledger per video recording provider, model, token count, image count, audio duration, and compute seconds consumed. At completion, each video has a precise cost-of-goods-sold figure broken down by pipeline stage. This granularity enables operators to identify which content types are most cost-efficient to produce, optimize prompt engineering to reduce token consumption, and negotiate volume discounts with providers based on actual usage data rather than estimates.

Video Assembly and Post-Production

The assembly engine operates on a timeline data structure that maps narration audio segments to their corresponding visual assets. Each segment specifies the visual type (still image, animated clip, or text card), the Ken Burns motion parameters (start/end crop, zoom direction, speed), the transition type to the next segment, and any text overlays with font, position, and timing. Background music is selected from a licensed library based on the script's mood annotations, with automatic ducking triggered by narration onset and offset timestamps. The renderer produces a final MP4 at 1080p/30fps with AAC audio, optimized for YouTube's recommended encoding profile.

Thumbnail Generation and Selection

The pipeline generates three thumbnail candidates per video using the script's thumbnail concepts as prompts. Each candidate is scored on four dimensions: visual contrast ratio (ensuring readability at thumbnail scale), color saturation (bright thumbnails earn higher click-through rates), face/object prominence (larger subjects perform better), and text overlay clarity (title text must remain legible at 320x180 pixels). The highest-scoring candidate is attached to the YouTube upload. A/B testing data from prior uploads feeds back into the scoring model, continuously improving thumbnail selection accuracy.

YouTube Upload Automation

The distribution module authenticates through YouTube Data API v3 using OAuth 2.0 with automatic token refresh. Uploads are resumable: if the connection drops mid-transfer, the module resumes from the last acknowledged byte rather than restarting. Video metadata includes AI-optimized titles (under 60 characters with power words), descriptions (structured with timestamps, key takeaways, and hashtags), and tags derived from the source content's topic modeling. The module sets the video's category, language, and default audio track, then schedules publication for the channel's optimal posting window based on historical audience activity data.

S3 Asset Archival and Cross-Platform Repurposing

Every intermediate and final asset is archived to S3 with a structured key hierarchy: /{channel}/{video-id}/{stage}/{asset}. This archive enables cross-platform repurposing: the same narration audio and visual assets can be re-composited into vertical-format shorts for TikTok and Instagram Reels, square-format clips for LinkedIn, and audio-only exports for podcast distribution. The archive also serves as a debugging surface: when a video's output quality is below expectations, operators can inspect every intermediate artifact to identify which pipeline stage introduced the issue.

Pipeline DAG entrypoint, stage routing, and cost ledger

Core stage modules: ingest, script, visual, voice, assembly

Batch runners, provider health checks, and upload helpers

Operational runbooks, failure playbooks, and API guides

Provider adapters with circuit breakers and fallback config

Script prompts, style directives, and metadata templates