arXiv AI
Oct 2

VideoEvolve: Evolving Agent Harnesses for Video Temporal Grounding

VideoEvolve is a framework that automatically evolves agent harnesses for video temporal grounding, a task that localizes events in videos based on natural-language queries. It introduces a Cloze-Structured Harness Representation to keep stage interfaces stable while allowing agent workflows and instructions to change, and uses Branch-Guided Harness Evolution to preserve promising code branches, guide local edits with execution feedback, and validate improvements. Experiments show that this automated evolution improves grounding performance across multiple benchmarks, with instruction refinement consistently yielding gains.

By Bingjun Luo, Yuhuan Fan, Jialin Guo, Siqi Li
arXiv Machine Learning
Jul 7

Incentivizing Vision Language Models to Search for Long Video Question Answering

arXiv:2607. 02959v1 Announce Type: cross Abstract: We introduce VSeek, an agentic framework that transforms long-video question answering (LVQA) from a passive, single-pass perception task into a multi-turn retrieval process.

By Harsh Goel, S P Sharan, Sahil Shah, Minkyu Choi, Joungbin An, Kristen Grauman, Sandeep P. Chinchali
arXiv Computer Vision
Sep 22

VideoGen-Agent: Reinforcing Video Generation Agents

VideoGen-Agent is a multimodal agent that uses multitask agentic reinforcement learning to coordinate external tools for video generation. It learns to augment, generate, and verify videos through multi‑turn interactions, guided by prompts and intermediate observations. On the new VABench benchmark, the agent improves base text‑to‑video performance by 19.1 points, and further upgrades to generation tools raise the score to 86.1, with human raters favoring the upgraded configuration in 84.3% of comparisons.

By Binxu Li, Haoyi Duan, Yuhui Zhang, Yaohui Zhang, Zihao Lin, Kaituo Feng, Suozhi Huang, Xiangyi Li, Yu Li, Chunyuan Li, Shilong Liu, Mengdi Wang