The paper surveys emerging video understanding agents that use video as the primary source and actively decide what information to inspect, retain, verify, and act upon. It formalizes an agent loop for video understanding and organizes the literature through a challenge‑to‑design taxonomy linking context bottlenecks, evidence sparsity, temporal causality, and multimodal ambiguity to corresponding design solutions. The survey also reviews state‑space paradigms, learning paradigms, supervision signals, benchmarks, evaluation protocols, and outlines open directions toward agentic‑native temporal modeling and video‑native agents.
By Xinyu Deng, Siwen Luo, Daochang Liu
arXiv:2608. 16316v1 Announce Type: cross Abstract: Large Multimodal Models (LMMs) for video reasoning have long been hindered by the high computational cost of processing vast amounts of visual information.
By Ao Shen, Yongheng Zhang, Yinghui Li, Manning Wang, Di Yin, Xing Sun
arXiv:2606. 07512v1 Announce Type: cross Abstract: Current Vision-Language Models struggle with hours-long videos because processing full-length visual sequences induces prohibitive token explosion and attention dilution.
By Cong Chen, Guo Gan, Kaixiang Ji, ChaoYang Zhang, Zhen Yang, Guangming Yao, Hao Chen, Jingdong Chen, Yi Yuan, Chunhua Shen
AgentVidBench is a new multi‑hop video question‑answering benchmark designed to evaluate spatial, temporal, and causal reasoning in multimodal large language models (MLLMs). Unlike existing tests that focus on simple scene queries or global summaries, AgentVidBench includes step‑by‑step solution traces to assess whether agents gather the necessary evidence to justify their answers. Experiments with 12 MLLMs show limited single‑turn performance, but integrating these models into agentic workflows improves both accuracy and trajectory scores, establishing AgentVidBench as a comprehensive testbed for future research on agentic video understanding.
By Seoyeon An, Hyeonseo Jang, Minsu Kim, Chanho Lee, Younghan Park, Kangwook Lee
arXiv:2609.15606v1 Announce Type: cross
Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable progress on short video understanding yet remain limited on long videos due to the...
By Weixin Xu, Zhenyu Yang, Bing Wang, Shengsheng Qian, Changsheng Xu
Recent advancements in chain-of-thought (CoT) reasoning have shown promise in enhancing video understanding and reasoning capabilities of multimodal large language models (MLLMs). However, existing CoT-based MLLMs require labor-intensive CoT annotations and incur substantial training and inference overhead.
The paper introduces OWLET, an on-policy weighted learning method for post-training Video Large Language Models. OWLET leverages a richer teacher view that combines global temporal context with question-conditioned local windows to provide visual evidence, and it employs evidence-aware credit redistribution and spatiotemporal state alignment to better utilize this evidence during reinforcement learning. The approach aims to improve video reasoning by aligning token-level supervision with spatiotemporal representations.
By Qitan Lv, Ke Fan, Hong Wang, Wen Wu, Yilin Wang, Bowen Zhou, Feng Wu, Chao Zhang
Video-OPSD introduces a post‑training framework for Video Large Language Models that leverages privileged visual evidence to enhance on‑policy self‑distillation. The method constructs a self‑teacher conditioned only on annotated evidence frames, while the student processes the full video, allowing the teacher to provide more focused supervision. Additionally, an evidence‑guided token optimization scheme weights distillation based on each token’s reliance on privileged evidence, improving perceptually grounded reasoning. Experiments demonstrate consistent gains over standard OPSD and comparable performance to GRPO with less training time.
By Ziyue Wang, Shiqi Huang, Weiwen Xu, Bihan Wen, Xudong Jiang
arXiv:2608.27871v1 Announce Type: new
Abstract: Long-video understanding remains challenging for Multimodal Large Language Models (MLLMs) due to limited context length. Uniform sampling may miss cruc...
By Ziling Huang, Shin'ichi Satoh
arXiv:2512. 05774v2 Announce Type: replace-cross Abstract: Long video understanding (LVU) is challenging because answering real-world queries often depends on sparse, temporally dispersed cues buried in hours of mostly redundant and irrelevant content.
By Ziyang Wang, Honglu Zhou, Shijie Wang, Junnan Li, Caiming Xiong, Silvio Savarese, Mohit Bansal, Michael S. Ryoo, Juan Carlos Niebles
arXiv:2609.09985v1 Announce Type: new
Abstract: Real-world video understanding requires integrating visual, audio, textual, and temporal evidence distributed across a video. Yet many pipelines use a...
By Sheng Li, Peng Liu, Qianqian Zhang, Tiancheng Zhao
arXiv:2607. 02959v1 Announce Type: cross Abstract: We introduce VSeek, an agentic framework that transforms long-video question answering (LVQA) from a passive, single-pass perception task into a multi-turn retrieval process.
By Harsh Goel, S P Sharan, Sahil Shah, Minkyu Choi, Joungbin An, Kristen Grauman, Sandeep P. Chinchali