arXiv Machine Learning

TLG: Temporal-Logic Grounding for Video Question Answering via Source-Annotation Reconstruction and Category-Targeted Reasoning

arXiv:2606. 01591v1 Announce Type: cross Abstract: The TimeLogic Challenge evaluates formal temporal-logic reasoning over video - 16 operators (before, after, until, since, always, co-occur, ordering, ...

arXiv Computer Vision
Sep 23

TEMPURA: Temporal Event Masked Prediction and Understanding for Reasoning in Action

arXiv:2505.01583v2 Announce Type: replace Abstract: Understanding causal event relationships and achieving fine-grained temporal grounding in videos remain challenging for vision-language models (VLM...

By Jen-Hao Cheng, Yi-Hao Peng, Huapeng Zhou, Vivian Wang, Huayu Wang, Hsiang-Wei Huang, Wenhao Chai, Hou-I Liu, Kuang-Ming Chen, Cheng-Yen Yang, Yi-Ling Chen, Vibhav Vineet, Qin Cai, Jenq-Neng Hwang
arXiv Computation and Language
Sep 25

STRAND: Benchmarking and Improving Object-Centric Spatio-Temporal Monitoring in Video Large Language Models

STRAND is a new benchmark that tests multimodal large language models’ ability to track objects, their states, and relationships over time in videos. It evaluates intermediate reasoning by breaking queries into sub‑questions and uses Faithful Accuracy to ensure all parts of an answer are correct. The authors also propose an object‑centric framework that builds structured trajectories and shows reduced hallucinations and better temporal consistency compared to existing models.

By Thong Nguyen, Tri Cao, Khoi Le, Cong-Duy Nguyen, Quynh Vo, See-Kiong Ng, Bryan Hooi Kuen-Yew
arXiv Computer Vision
Sep 1

Improving Spatial-Temporal Reasoning in Video-Language Models with Structured Video Prompting

The paper introduces structured video prompting, a training‑free inference‑time technique that augments input videos with lightweight spatial and temporal structure to provide explicit anchors for evidence organization. By applying this method to two video benchmarks and two open video‑language models, the authors demonstrate performance improvements across several tasks, with gains varying by model and task. The study suggests that failures in video‑language models stem not only from reasoning capacity but also from how video evidence is presented during inference.

By Sadegh Mohammadian
arXiv AI
Sep 10

TimeBlind: A Spatio-Temporal Compositionality Benchmark for Video LLMs

TimeBlind is a diagnostic benchmark designed to evaluate fine‑grained spatio‑temporal compositionality in video large language models (LLMs). It categorizes temporal understanding into three levels—atomic event recognition, event property characterization, and reasoning about event interdependencies—and uses a minimal‑pairs paradigm where video pairs share identical static content but differ only in temporal structure. Across 20 state‑of‑the‑art MLLMs tested on 600 curated instances, the best model achieved only 48.2% instance accuracy, far below human performance of 98.2%, highlighting a reliance on static visual shortcuts rather than true temporal reasoning.

By Baiqi Li, Kangyi Zhao, Ce Zhang, Chancharik Mitra, Jean de Dieu Nyandwi, Gedas Bertasius
arXiv Computer Vision
Aug 26

RT-NeuS: Towards Real-Time Neuro-Symbolic Video Understanding via Adaptive Temporal Verification

RT-NeuS is a neuro‑symbolic framework for long‑form video question answering that retains the accuracy and formal guarantees of temporal‑logic‑guided methods while dramatically reducing inference latency. It achieves this by using coarse‑to‑fine adaptive sampling to focus on query‑relevant frames and batched proposition detection with KV‑cache reuse, enabling all propositions to be evaluated in a single forward pass. Experiments on LongVideoBench, Video‑MME, and MLVU show up to a 13× speed‑up on an NVIDIA H200 GPU while matching or surpassing prior neuro‑symbolic accuracy.

By Shawn Liang, Sahil Shah, Chengwei Zhou, S P Sharan, Harsh Goel, Arnab Sanyal, Sandeep Chinchali, Gourav Datta
arXiv AI
Aug 20

Event-Causal RAG: A Retrieval-Augmented Generation Framework for Long Video Reasoning in Complex Scenarios

Event-Causal RAG (EC‑RAG) is a lightweight retrieval‑augmented framework designed for reasoning over ultra‑long and streaming videos. It segments video streams into semantically complete events using a dual visual‑audio sentinel mechanism, representing each event as a State‑Event‑State (SES) structure that captures pre‑event, event, and post‑event states. During question answering, bidirectional graph retrieval accesses relevant predecessor and successor events from a dual vector‑graph memory, and answers are generated using both this structured memory and the corresponding video evidence. The authors also introduce ECV‑1H, an hour‑scale long‑video QA benchmark with over 150 hours of untrimmed video and 1,251 human‑annotated QA pairs, where EC‑RAG achieves significant accuracy gains across multiple video foundation models while maintaining efficient streaming memory usage on a single RTX 5090 GPU.

By Peizheng Yan, Yu Zhao, Liang Xie, Juntong Qi, Mingming Wang, Erwei Yin