arXiv:2610.00757v1 Announce Type: new
Abstract: Long-video question answering is limited by the high cost of visual tokens and by the fixed context width of current VLMs. A long-video question may re...
By Haowen Guan, Shengzhi Li, Shichao Pei
arXiv:2609.16722v1 Announce Type: new
Abstract: Scaling Multimodal Large Language Models (MLLMs) to long-form video understanding is bottlenecked by the explosion of visual tokens, which saturates co...
By Haoyu Guo, Yuan Feng, Junlin Lv, Mingjun Xiao, S Kevin Zhou, Xike Xie
arXiv:2608.27871v1 Announce Type: new
Abstract: Long-video understanding remains challenging for Multimodal Large Language Models (MLLMs) due to limited context length. Uniform sampling may miss cruc...
By Ziling Huang, Shin'ichi Satoh
MemoryCard is a video-memory-based augmentation framework designed to improve long-video question answering for Vision‑Language Models. It segments lengthy videos into semantically coherent units—each representing a distinct topic or event—by performing a self‑reading process over the video and aligned utterances. For each unit, the framework generates an event‑level video gist and selects representative visual moments, which are compiled into unified Memory Cards that are used for retrieval and answering questions, yielding up to a 21.8% relative accuracy improvement under comparable visual‑token budgets.
By Qing Yang, Pengcheng Huang, Xinze Li, Zhenghao Liu, Yukun Yan, Yu Gu, Ge Yu, Gang Li, Maosong Sun
The paper introduces S‑MME, a benchmark for diagnosing shortcut‑biased behavior in Multimodal Large Language Models (MLLMs) on long‑video question answering. It proposes CLEAR, a training‑free framework that estimates and mitigates bias by treating the full video‑question pair as a complete view and intervening on hidden states during inference. Experiments on five benchmarks demonstrate that CLEAR improves prediction accuracy and robustness across different MLLMs.
By Xusheng Liang, Jianqiao Sun, Hao Zhang, Yulei Niu, Hengshuang Zhao, Jiawei Ma
arXiv:2608.05592v2 Announce Type: replace
Abstract: Multimodal Large Language Models (MLLMs) have made strong progress in video understanding, yet long videos remain difficult: the visual token budge...
By Ziling Huang, Shin'ichi Satoh
arXiv:2607. 24794v1 Announce Type: new Abstract: While Multimodal Large Language Models (MLLMs) demonstrate superior generalization in fundamental video tasks, restricted context windows limit their long video understanding.
By Linghao Meng, Qiankun Li, Junyuan Mao, Pujin Liao, Zhicheng He, Enbo Zhang, Kun Wang, Yang Liu, Huazhu Fu, Yueming Jin
Scaling Multimodal Large Language Models (MLLMs) to long-form video understanding is bottlenecked by the explosion of visual tokens, which saturates context windows and incurs prohibitive costs. Curre...
LDDR (Linear DPP Dynamic Resolution) is a training‑free, budget‑aware video frame sampling framework designed for multimodal large language models (MLLMs). It uses query‑aware Determinantal Point Process (DPP) selection in a task‑conditioned feature space, achieving a 3× runtime speedup over standard DPP baselines. By assigning a non‑uniform token budget—allocating more tokens to informative, non‑redundant frames and downscaling or pruning less useful ones—LDDR improves performance by 2.5 points in low‑budget settings and 1.6 points in high‑budget scenarios across four video benchmarks and multiple MLLM families.
By Jingfeng Chen, Jiawen Qian, Wendi Deng, Yinuo Guo, Jiaqi Yu, Sicong Leng, Raghuveer Thirukovalluru, Bhuwan Dhingra
VideoLVR introduces a method for multimodal large language models to maintain visual grounding during chain-of-thought reasoning by constructing latent visual evidence from local video regions at relevant moments. It uses a visual token prediction objective that unifies visual evidence reconstruction and text generation, allowing models to interleave text tokens with latent visual states. The approach is further extended with reinforcement learning, treating visual tokens as actions and optimizing policies with final-answer rewards, thereby improving video understanding without expanding the multimodal context.
By Shilong Li, Xingyuan Bu, Hao Shi, Yun Xing, Weixin Feng, Qiangpeng Yang, Shilei Wen
Recent advancements in MLLM-based long-form video understanding have mitigated inference-time computational cost and limited context lengths by selecting query-relevant frames. However, existing approaches predominantly rely on external proxy scorers and rigid heuristic rules, inevitably suffering from misalignment with the target MLLM's intrinsic evidence and failing to accommodate the non-uniform spatiotemporal information density.
arXiv:2609.37042v1 Announce Type: cross
Abstract: Video Large Language Models (VideoLLMs) have achieved strong video understanding capabilities but incur substantial inference overhead due to the lar...
By Shuo Yang, Changbai Li, Rui Tang, Xinyu Zhao, Linlin Yang, Baochang Zhang