TimeBlind is a diagnostic benchmark designed to evaluate fine‑grained spatio‑temporal compositionality in video large language models (LLMs). It categorizes temporal understanding into three levels—atomic event recognition, event property characterization, and reasoning about event interdependencies—and uses a minimal‑pairs paradigm where video pairs share identical static content but differ only in temporal structure. Across 20 state‑of‑the‑art MLLMs tested on 600 curated instances, the best model achieved only 48.2% instance accuracy, far below human performance of 98.2%, highlighting a reliance on static visual shortcuts rather than true temporal reasoning.
By Baiqi Li, Kangyi Zhao, Ce Zhang, Chancharik Mitra, Jean de Dieu Nyandwi, Gedas Bertasius
arXiv:2504. 01407v3 Announce Type: replace-cross Abstract: Long video understanding poses a fundamental challenge for large video-language models (LVLMs) due to the overwhelming number of frames and the risk of losing essential context through naive downsampling.
By Yuan Zhang, Junwen Pan, Rui Zhang, Xin Wan, Qizhe Zhang, Ming Lu, Qi She, Shanghang Zhang
arXiv:2606. 13141v1 Announce Type: new Abstract: Retrieval-augmented generation is moving beyond text into long, egocentric video, where systems must select query-relevant chunks across multiple modalities and temporal granularities.
By Yuho Lee, Jisu Shin, Nicole Hee-Yeon Kim, Jihwan Bang, Juntae Lee, Kyuwoong Hwang, Fatih Porikli, Hwanjun Song
Event-Causal RAG (EC‑RAG) is a lightweight retrieval‑augmented framework designed for reasoning over ultra‑long and streaming videos. It segments video streams into semantically complete events using a dual visual‑audio sentinel mechanism, representing each event as a State‑Event‑State (SES) structure that captures pre‑event, event, and post‑event states. During question answering, bidirectional graph retrieval accesses relevant predecessor and successor events from a dual vector‑graph memory, and answers are generated using both this structured memory and the corresponding video evidence. The authors also introduce ECV‑1H, an hour‑scale long‑video QA benchmark with over 150 hours of untrimmed video and 1,251 human‑annotated QA pairs, where EC‑RAG achieves significant accuracy gains across multiple video foundation models while maintaining efficient streaming memory usage on a single RTX 5090 GPU.
By Peizheng Yan, Yu Zhao, Liang Xie, Juntong Qi, Mingming Wang, Erwei Yin
arXiv:2606. 06294v1 Announce Type: cross Abstract: Temporal Grounding (TG) aims to localize video segments corresponding to a textual query.
By Qi Xu, Yue Tan, Shihao Chen, Jiahao Meng, Anna Wang, Shunping Ji, Hao Fei, Jason Li
We introduce Predictive State Retrieval (PSR), a task in which a model observes a short video prefix and a temporal question about an object's future state, then retrieves instances from other videos or images that depict that state. Unlike action anticipation, which predicts a label, moment retrieval, which localizes an observed event within a video, or video generation, which synthesizes pixels, PSR combines anticipation with cross-instance retrieval across multiple temporal horizons.
arXiv:2607. 24794v1 Announce Type: new Abstract: While Multimodal Large Language Models (MLLMs) demonstrate superior generalization in fundamental video tasks, restricted context windows limit their long video understanding.
By Linghao Meng, Qiankun Li, Junyuan Mao, Pujin Liao, Zhicheng He, Enbo Zhang, Kun Wang, Yang Liu, Huazhu Fu, Yueming Jin
Concord introduces a Video Relational Algebra (VRA) that models videos, transcripts, frames, and object tracks, enabling semantic video queries. It applies approximate optimizations to rewrite VRA queries, reducing large language model (MLLM) usage by processing transcripts or using detection and tracking instead of full-video MLLM joins. Experiments on soccer broadcasts and lectures show that Concord sends only a small fraction of video to the MLLM, cutting costs by up to 87%, and improves cross‑camera query accuracy from an F1 of .364 to .813 without any MLLM calls.
By Sultan Muratbek, Charisse Ivana Yeung, Chanwut Kittivorawong, Alvin Cheung
arXiv:2602. 14612v4 Announce Type: replace-cross Abstract: Answering natural-language questions over multi-hour audio requires both event recognition and temporal grounding.
By Kartik Hegde, Arvind Krishna Sridhar, Naveen Vakada, Yinyi Guo, Erik Visser
arXiv:2608. 07663v1 Announce Type: cross Abstract: When videos extend from hours to days, directly processing them end-to-end becomes impractical for current Multi-modal Large Language Models (MLLMs).
By Yeeun Choi, Youngbeom Yoo, Joon-Young Lee, Hyolim Kang, Seon Joo Kim
arXiv:2607. 25266v2 Announce Type: replace-cross Abstract: Multimodal large language models (MLLMs) have enabled long-form video understanding at a scale that was not previously possible.
By Ghazal Kaviani, Ghassan AlRegib
The paper introduces Linguistic Trajectory Encoding (LTE), a hybrid representation that compresses dynamic object motion histories using natural language descriptions, sparse spatial anchors, and visual anchors. LTE adapts compression to motion complexity by anchoring periods without reliable observations to the last seen location while preserving accuracy with geometric waypoints and linguistic descriptions. Evaluated on the newly constructed Spatial Memory Benchmark (SMB) from EgoLife multi‑day recordings, LTE achieves 45.3 % success in semantic trajectory retrieval and 48.7 % in long‑horizon object retrieval, outperforming prior structured‑memory and VLM baselines, and compresses trajectories 8.7×–26.1× with sub‑second query latency on 24‑hour video.
By Tianyidan Xie, Shenyi Wang, Qiang Tang, Mingjie Wang, Zhicheng Qiu, Xuanfu Li, Zhan Xu, Jian Yang, Lanjun Wang, Zili Yi