VideoGen-Agent: Reinforcing Video Generation Agents
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
VideoGen-Agent is a multimodal agent that uses multitask agentic reinforcement learning to coordinate external tools for video generation. It learns to augment, generate, and verify videos through multi‑turn interactions, guided by prompts and intermediate observations. On the new VABench benchmark, the agent improves base text‑to‑video performance by 19.1 points, and further upgrades to generation tools raise the score to 86.1, with human raters favoring the upgraded configuration in 84.3% of comparisons.
Video-Coder is a novel code-driven video generation agent that autonomously writes and executes code to build dynamic 3D scenes, providing structure and motion cues to downstream video generators. The authors introduce a dedicated data pipeline and curate two high‑quality datasets—Video‑Coder‑SFT‑6k and Video‑Coder‑RL‑2k—along with the Video‑Coder Harness, an interactive environment that integrates code execution, 3D rendering, and feedback for training. Using a combination of supervised fine‑tuning and harness‑native agentic reinforcement learning with dual rewards, Video‑Coder achieves notable performance gains, improving Minimax‑H3 by about 7 points on PhyGenBench and 24 points on VBench‑2.0.
VideoTIR introduces a reinforcement‑learning approach to improve long‑video understanding by encouraging multimodal large language models to use comprehensive multi‑level toolkits efficiently. It combines Zero‑RL and SFT cold‑starting strategies to help models retrieve and focus on meaningful video segments, images, and regions, thereby reducing hallucinations. The method includes Toolkit Action Grouped Policy Optimization (TAGPO) to streamline tool‑calling and a sandbox‑based trajectory synthesis framework for high‑quality data, achieving strong results on three long‑video QA benchmarks.
The paper surveys emerging video understanding agents that use video as the primary source and actively decide what information to inspect, retain, verify, and act upon. It formalizes an agent loop for video understanding and organizes the literature through a challenge‑to‑design taxonomy linking context bottlenecks, evidence sparsity, temporal causality, and multimodal ambiguity to corresponding design solutions. The survey also reviews state‑space paradigms, learning paradigms, supervision signals, benchmarks, evaluation protocols, and outlines open directions toward agentic‑native temporal modeling and video‑native agents.
ParaVT introduces a multi‑agent reinforcement learning framework that enables parallel video‑tool calling, allowing multiple time‑window crops to be processed in a single turn. The authors identify a Tool Prior Paradox where inherited tool‑use priors both aid exploration and conflict with supervised output formatting, leading to potential tool skipping. They propose PARA‑GRPO, which adds a format reward and randomizes the overview‑frame budget to encourage proper tool use, achieving a 7.9% performance gain over the Qwen3‑VL baseline on six long‑video benchmarks.
VideoWeaver is an agent harness and benchmark for long‑video generation that evaluates and evolves reusable skills. It allows an agent to dynamically compose foundation skills into a workflow based on a single high‑level instruction, rather than following a static pipeline. The benchmark includes 16 task categories and 285 cases with multimodal references, and an evidence‑grounded agent‑as‑judge inspects execution traces and final videos to guide skill evolution, improving both process and output quality.