arXiv:2608. 20127v1 Announce Type: new Abstract: Video Temporal Grounding (VTG) faces significant challenges when natural language queries must distinguish between multiple events involving visually similar entities, particularly when relying on fine-grained visual attributes that are difficult to describe accurately in words alone.
TimeBlind is a diagnostic benchmark designed to evaluate fine‑grained spatio‑temporal compositionality in video large language models (LLMs). It categorizes temporal understanding into three levels—atomic event recognition, event property characterization, and reasoning about event interdependencies—and uses a minimal‑pairs paradigm where video pairs share identical static content but differ only in temporal structure. Across 20 state‑of‑the‑art MLLMs tested on 600 curated instances, the best model achieved only 48.2% instance accuracy, far below human performance of 98.2%, highlighting a reliance on static visual shortcuts rather than true temporal reasoning.
The paper introduces Temporally-Grounded Language Generation (TGLG), a benchmark that tests vision‑language models on their ability to produce semantically accurate and temporally precise utterances in real‑time settings. It identifies perceptual updating and contingency awareness as key capabilities, curates datasets from sports broadcasting and egocentric interactions, and proposes the TRACE metric to jointly evaluate semantic similarity and temporal alignment. The authors also present VLM‑TSI, a model that interleaves visual and linguistic tokens in a time‑synchronized manner, achieving better performance than a strong baseline yet still showing modest overall results, underscoring the challenge of real‑time VLMs.
ActionPiece rethinks how actions are tokenized for autoregressive vision‑language‑action models by introducing physical rank consistency (PRC) to evaluate relational fidelity of reconstructed actions. The method jointly supervises representation learning and quantization to preserve local physical distance rankings, improving both PRC and policy success. Experiments on LIBERO, LIBERO‑Plus, SimplerEnv, and VLA‑Arena show significant gains over baseline tokenizers.
arXiv:2606. 06294v1 Announce Type: cross Abstract: Temporal Grounding (TG) aims to localize video segments corresponding to a textual query.
WALL-WM is a World Action Model that shifts video-action learning from chunk-centric optimization to event-grounded Vision-Language-Action pretraining, using semantically coherent action events as the atomic unit of learning. Existing WAMs commonly initialize from multimodal or video foundation models and then optimize fixed-length action chunks conditioned directly on the current observation and instruction.
arXiv:2608.30584v1 Announce Type: new Abstract: Despite the impressive progress of recent MLLMs on spatio-temporal video grounding (STVG), existing evaluations and training data focus primarily on si...
arXiv:2607. 16322v1 Announce Type: cross Abstract: Micro-gesture recognition demands the detection of fleeting, spatially localized movements that are frequently overwhelmed by dominant static appearances and background noise.
Event-Causal RAG (EC‑RAG) is a lightweight retrieval‑augmented framework designed for reasoning over ultra‑long and streaming videos. It segments video streams into semantically complete events using a dual visual‑audio sentinel mechanism, representing each event as a State‑Event‑State (SES) structure that captures pre‑event, event, and post‑event states. During question answering, bidirectional graph retrieval accesses relevant predecessor and successor events from a dual vector‑graph memory, and answers are generated using both this structured memory and the corresponding video evidence. The authors also introduce ECV‑1H, an hour‑scale long‑video QA benchmark with over 150 hours of untrimmed video and 1,251 human‑annotated QA pairs, where EC‑RAG achieves significant accuracy gains across multiple video foundation models while maintaining efficient streaming memory usage on a single RTX 5090 GPU.
Action Quality Assessment (AQA) aims to objectively evaluate performance quality from action videos. Most existing methods follow a ``one-by-one'' paradigm, training a separate model for each action type.
Video captioning requires fine-grained spatio-temporal understanding of videos, including spatial perception of where objects are located and temporal perception of when events occur. Existing MLLMs usually generate captions directly from video inputs without exposing the perceptual evidence behind descriptions.
The paper introduces Parallel Tube Decoding (PTD), a generative approach for spatio‑temporal video grounding that splits the task into a temporal block and simultaneous time‑conditioned spatial blocks, eliminating token‑level and trajectory‑level dependencies. PTD uses Decoupled Block Attention to allow parallel spatial generation while maintaining shared video‑query context, and incorporates localization‑aware policy optimization for temporal boundaries and spatial geometry. Experiments on VidSTG show PTD cuts tube completion latency by 79× and boosts spatial decoding throughput by 92× compared to autoregressive decoding, while improving grounding accuracy and performing well on related tasks such as temporal grounding, VideoQA, and referring video object tracking.