arXiv AI

Bridging VideoQA and Video-Guided Agentic Tasks via Generalized Keyframe Extraction

arXiv:2606. 29445v1 Announce Type: cross Abstract: Video understanding is a fundamental capability for multimodal intelligence, and recent Multimodal Large Language Models (MLLMs) have achieved remarkable performance on Video Question Answering (VideoQA) benchmarks.

arXiv AI
Sep 21

AgentVidBench: A Multi-Hop Video Question Answering Benchmark for Evaluating MLLM Agents

AgentVidBench is a new multi‑hop video question‑answering benchmark designed to evaluate spatial, temporal, and causal reasoning in multimodal large language models (MLLMs). Unlike existing tests that focus on simple scene queries or global summaries, AgentVidBench includes step‑by‑step solution traces to assess whether agents gather the necessary evidence to justify their answers. Experiments with 12 MLLMs show limited single‑turn performance, but integrating these models into agentic workflows improves both accuracy and trajectory scores, establishing AgentVidBench as a comprehensive testbed for future research on agentic video understanding.

By Seoyeon An, Hyeonseo Jang, Minsu Kim, Chanho Lee, Younghan Park, Kangwook Lee
arXiv AI
Sep 15

Evaluation of MLLM-Agnostic Plug-and-Play Keyframe Selection Methods for Long Video Understanding

The paper evaluates five training‑free, plug‑and‑play keyframe selection methods for multimodal large language models (MLLMs) on long‑video understanding tasks. It compares these methods across three different MLLMs and three video question‑answering benchmarks, finding that QAaF performs best in 13 of 15 settings while FOCUS ranks second. The study offers a unified benchmark for assessing MLLM‑agnostic keyframe selection techniques.

By Dilip Sarkar, Md. Safayet Islam, Liang Liang
arXiv Computer Vision
2d ago

FocusGraph: Graph-Structured Frame Selection for Embodied Long Video Question Answering

arXiv:2603.04349v2 Announce Type: replace Abstract: Understanding long videos is crucial for embodied intelligent agents, as their performance depends on effectively accumulating and using long-horiz...

By Tatiana Zemskova, Solomon Andryushenko, Ilya Obrubov, Viktoriia Khoruzhaia, Ekaterina Eroshenko, Ekaterina Derevyanka, Dmitry Yudin
arXiv AI
Jun 12

ReFoCUS: Reinforcement-guided Frame Optimization for Contextual Understanding

arXiv:2506. 01274v2 Announce Type: replace-cross Abstract: Recent progress in Large Multi-modal Models (LMMs) has enabled effective vision-language reasoning, yet the ability to video understanding remains constrained by suboptimal frame selection strategies, albeit with the rapid development of video-specialized LMMs.

By Hosu Lee, Junho Kim, Hyunjun Kim, Yong Man Ro
arXiv Machine Learning
Jul 7

Incentivizing Vision Language Models to Search for Long Video Question Answering

arXiv:2607. 02959v1 Announce Type: cross Abstract: We introduce VSeek, an agentic framework that transforms long-video question answering (LVQA) from a passive, single-pass perception task into a multi-turn retrieval process.

By Harsh Goel, S P Sharan, Sahil Shah, Minkyu Choi, Joungbin An, Kristen Grauman, Sandeep P. Chinchali
arXiv Computer Vision
Sep 10

VideoTIR: Accurate Understanding for Long Videos with Efficient Tool-Integrated Reasoning

VideoTIR introduces a reinforcement‑learning approach to improve long‑video understanding by encouraging multimodal large language models to use comprehensive multi‑level toolkits efficiently. It combines Zero‑RL and SFT cold‑starting strategies to help models retrieve and focus on meaningful video segments, images, and regions, thereby reducing hallucinations. The method includes Toolkit Action Grouped Policy Optimization (TAGPO) to streamline tool‑calling and a sandbox‑based trajectory synthesis framework for high‑quality data, achieving strong results on three long‑video QA benchmarks.

By Zhe Gao, Shiyu Shen, Taifeng Chai, Weinong Wang, Haotian Xu, Xing Wu, Wenbin Li, Qi Fan, Yang Gao, Dacheng Tao