DramaChain Bench is the first benchmark that evaluates every stage of short‑drama production—from script to finished video—using a unified dimension system called DramaChain Dimensions. It incorporates three in‑house systems: a calibrated agent that mirrors commercial workflows, a labeling system with 5,785 items scored by professional annotators, and an agentic judge that automatically scores leaf dimensions with high correlation to human judgments. The benchmark demonstrates that defects in early stages cascade through the pipeline, showing that final episode quality depends on more than just video generation.
arXiv:2605.22144v2 Announce Type: replace Abstract: Existing approaches for digital short-drama production typically rely on one-shot LLM generated scripts and loosely coupled pipelines, which fail t...
arXiv:2607. 24241v1 Announce Type: cross Abstract: Progress in video generation keeps narrowing the visual gap between AI-generated and professionally produced footage, yet most benchmarks still draw prompts from web sources or LLM templates and score them with untrained, generic multimodal models.
Progress in video generation keeps narrowing the visual gap between AI-generated and professionally produced footage, yet most benchmarks still draw prompts from web sources or LLM templates and score them with untrained, generic multimodal models. More fundamentally, their evaluation taxonomies remain rudimentary (overall visual quality, coarse text alignment and temporal smoothness) rather than the professional Cinematic Language criteria by which films are actually made and judged, so they assess basic video plausibility rather than film-grade craft.
Timeline-Bench is a benchmark comprising 56 real video‑editing tasks that require AI agents to transform raw production material into finished videos. Each task includes a brief, source assets, a container, and a set of tests that assess format, content, brief compliance, and quality based on 2,582 blind judgments by 43 video editors. In evaluations, the best agent resolved only 15 of the 56 tasks, and most failures were due to quality tests rather than technical errors.
VibeVideoBench is a benchmark that evaluates multimodal agents capable of generating videos by writing code to render each frame and sound. The benchmark contains 239 self‑contained tasks in English and Chinese across 12 categories, requiring agents to produce a finished MP4 with sound within one hour without using any pre‑trained image or video generation model. Evaluation is hierarchical, using program checks, 13k task‑specific rubric checks, and five quality dimensions scored by three VLM judge panels, and it shows that GPT‑6 Astra and Claude Opus 5.5 perform best among nine frontier agents.
arXiv:2608.29621v1 Announce Type: cross Abstract: Long-horizon story-driven video generation requires a production agent to coordinate narrative decomposition, state tracking, shot design, prompt con...
VideoWeaver is an agent harness and benchmark for long‑video generation that evaluates and evolves reusable skills. It allows an agent to dynamically compose foundation skills into a workflow based on a single high‑level instruction, rather than following a static pipeline. The benchmark includes 16 task categories and 285 cases with multimodal references, and an evidence‑grounded agent‑as‑judge inspects execution traces and final videos to guide skill evolution, improving both process and output quality.
arXiv:2610.35864v1 Announce Type: new Abstract: Diffusion video models produce striking clips, but they cannot reliably render an exact formula, a correct chart, a ruleconsistent game or narration sy...
Diffusion video models produce striking clips, but they cannot reliably render an exact formula, a correct chart, a rule-consistent game or narration synchronized to on-screen events. A different rout...
arXiv:2605. 21917v2 Announce Type: replace-cross Abstract: Training Vision Language Models (VLMs) for video event reasoning requires high-quality structured annotations capturing not only what happened, but when, where, why, and with what consequence, at a scale manual labelling cannot support.
arXiv:2606. 24107v1 Announce Type: cross Abstract: Short dramas, with their rapid shot rhythms, dialogue-driven focus shifts, and demanding cinematographic grounding, pose challenges that prompt-level or text-only video generation pipelines struggle to meet.
arXiv:2604. 25220v2 Announce Type: replace Abstract: Data videos combine animated visualizations with synchronized narration to communicate quantitative information and are widely used in journalism, education, and public communication.