The paper introduces Parallel Tube Decoding (PTD), a generative approach for spatio‑temporal video grounding that splits the task into a temporal block and simultaneous time‑conditioned spatial blocks, eliminating token‑level and trajectory‑level dependencies. PTD uses Decoupled Block Attention to allow parallel spatial generation while maintaining shared video‑query context, and incorporates localization‑aware policy optimization for temporal boundaries and spatial geometry. Experiments on VidSTG show PTD cuts tube completion latency by 79× and boosts spatial decoding throughput by 92× compared to autoregressive decoding, while improving grounding accuracy and performing well on related tasks such as temporal grounding, VideoQA, and referring video object tracking.
By Hanoona Rasheed, Haania Siddiqui, Ming-Hsuan Yang, Fahad Shahbaz Khan, Salman Khan
TempoGround is a vision‑language model–native framework for streaming visual grounding that detects cross‑frame object correspondence and explicitly models object presence states. It uses a curriculum prediction mechanism to resolve 2D instance association, predict object entry, continuation, or exit, decode 2D boxes, and lift them to 3D camera‑frame boxes. The approach is further refined with Streaming Grounding Reinforcement, which optimizes grounding, identity, and consistency rewards, and achieves significant improvements on multiple streaming visual grounding benchmarks.
By Leqian Ding, Junning Qiu, Manwen Yang, Yu Guo, Fei Wang
arXiv:2609.16722v1 Announce Type: new
Abstract: Scaling Multimodal Large Language Models (MLLMs) to long-form video understanding is bottlenecked by the explosion of visual tokens, which saturates co...
By Haoyu Guo, Yuan Feng, Junlin Lv, Mingjun Xiao, S Kevin Zhou, Xike Xie
arXiv:2607. 13421v1 Announce Type: cross Abstract: Spatio-Temporal Video Grounding (STVG) aims to retrieve the visual trajectory of a specific object from a video stream as described by a natural language expression.
By Kai Chen, Ming Dai, Wenxuan Cheng, Wankou Yang
arXiv:2607. 02269v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) have demonstrated immense promise in Spatio-Temporal Video Grounding (STVG).
By Rintaro Otsubo, Ryo Fujii, Reina Ishikawa, Taiki Kanaya, Kanta Sawafuji, Hiroki Kajita, Shigeki Sakai, Hideo Saito, Ryo Hachiuma
WALL-WM is a World Action Model that shifts video-action learning from chunk-centric optimization to event-grounded Vision-Language-Action pretraining, using semantically coherent action events as the atomic unit of learning. Existing WAMs commonly initialize from multimodal or video foundation models and then optimize fixed-length action chunks conditioned directly on the current observation and instruction.
Pocket-STVG (P-STVG) is a lightweight cascade architecture for Spatio-Temporal Video Grounding that combines efficient pre‑trained components: a temporal‑aware video encoder based on MobileViCLIP, a spatial encoder‑decoder from MDETR, and a shared aligned text encoder. Temporal localization is achieved with a lightweight 1D U‑Net or a simple thresholding strategy, allowing the model to work in both weakly supervised and zero‑shot settings. With fewer than 90 M parameters, P-STVG matches or surpasses prior weakly supervised and zero‑shot methods while offering a more memory‑ and compute‑efficient pipeline for large‑scale video collections.
By Alberto Presta, Michal Byra, Grzegorz Stefa\'nski, Karol Szurkowski, Eryk Ko{\l}odziejczyk, Krzysztof Arendt
arXiv:2608.20473v1 Announce Type: new
Abstract: Video language models process videos as dense visual-token sequences with substantial representational redundancy. Compressing these sequences is there...
By Wenti Yin, Xiaotian Han, Junyuan Shang, Yuchen Ding, Shuohuan Wang, Dianhai Yu, Changxin Gao, Nong Sang
ShallowStream is a framework for streaming video understanding that uses the shallow layers of a multimodal large language model (MLLM) to encode frames and build a lightweight index. During streaming, it maintains an always‑on index via the KV cache of shallow layers, and at query time it scores context frames using shallow‑layer attention and selects diverse evidence for answering. The approach matches the performance of leading streaming methods while cutting per‑frame prefill latency and 10‑second end‑to‑end latency by up to 52.1× and 11.9×, respectively.
By Jitai Hao, Ke Yang, Qiang Huang, Jun Yu
arXiv:2607.14935v2 Announce Type: replace
Abstract: Recent advances in video understanding have spanned motion, long video, and streaming interaction, driving this field toward real-world application...
By Xinhao Li, Yuhan Zhu, Xiangyu Zeng, Yuhao Dong, Haoning Wu, Zhiqiu Zhang, Yuandong Yang, Changlian Ma, Qingyu Zhang, Yansong Shi, Xinyu Chen, Haoran Chen, Zizheng Huang, Jun Zhang, Kun Ouyang, Lin Sui, Ziang Yan, Yicheng Xu, Chenting Wang, Yinan He, Hongjie Zhang, Yi Wang, Yu Qiao, Yali Wang, Ziwei Liu, Kai Chen, Limin Wang
SYNCR is a synthetic benchmark designed to evaluate multimodal large language models on cross‑video reasoning. It contains 4,000 question‑answer pairs across 4,827 unique videos, covering tasks in temporal alignment, spatial tracking, comparative reasoning, and holistic synthesis. The benchmark reveals a significant performance gap between current models and humans, with models excelling at temporal ordering but struggling with precise physical and spatial reasoning.
By Sara Ghazanfari, Siddharth Garg, Prashanth Krishnamurthy, Farshad Khorrami
arXiv:2609.39021v1 Announce Type: new
Abstract: Reinforcement learning (RL) has substantially improved the reasoning ability of multimodal language models through verifiable rewards and increasingly...
By Haiying He, Xin Zheng, Shaoli Hu, Shijun Xiao, Xuanhe Liu, Bing Li, Harry Yang