arXiv AI

Kairos: A Dataset for Fine-Grained Video-Language Modeling over Space, Time, and Dynamics

Kairos is a new video dataset designed for fine-grained video-language modeling, featuring long-duration videos from ten minutes to half an hour. Each video is annotated with time-resolved labels that capture ongoing actions, entity appearances, attributes, interactions, and evolving contextual cues throughout the timeline. The dataset supports fine-grained evaluation, long-range modeling, reasoning, instruction data construction, representation learning, and video generation.

arXiv Computer Vision
Sep 23

TEMPURA: Temporal Event Masked Prediction and Understanding for Reasoning in Action

arXiv:2505.01583v2 Announce Type: replace Abstract: Understanding causal event relationships and achieving fine-grained temporal grounding in videos remain challenging for vision-language models (VLM...

By Jen-Hao Cheng, Yi-Hao Peng, Huapeng Zhou, Vivian Wang, Huayu Wang, Hsiang-Wei Huang, Wenhao Chai, Hou-I Liu, Kuang-Ming Chen, Cheng-Yen Yang, Yi-Ling Chen, Vibhav Vineet, Qin Cai, Jenq-Neng Hwang
arXiv Computer Vision
Sep 4

The Shape of Time: Video-Token Contrast for Temporal Understanding in VideoLMs

The paper introduces VT-Contrast, a representation-level temporal counterfactual objective designed to improve temporal understanding in Video Language Models (VideoLMs). By supervising late-layer last-frame video tokens and contrasting order-preserving views with reordered counterfactuals graded by Kendall tau distance, VT-Contrast addresses the mismatch between ordered video input and text-based supervision. The method requires no architectural changes, is compatible with various VideoLM training tasks, and demonstrates improved performance on temporal understanding benchmarks.

By Yumeng Shi, Quanyu Long, Yin Wu, Wenya Wang
arXiv Computer Vision
Aug 31

Training-Free Temporal Abstraction for General Video Understanding

The paper introduces STITCH, a training‑free method that partitions videos into semantically meaningful temporal chunks using a frozen video‑text backbone. By detecting changes in the embedding sequence of short video windows, STITCH produces reusable temporal abstractions that can be applied to multiple tasks such as event boundary detection, language‑based moment retrieval, and frame selection for vision‑language models. Experiments show that STITCH performs competitively with specialized methods while requiring no task‑specific training, especially when processing is limited to a few frames or tokens.

By Etienne Casanova, Sevan Brodjian, Pietro Perona
arXiv AI
Jul 20

LVSum: A Benchmark for Timestamp-Aware Long Video Summarization

arXiv:2604. 10024v2 Announce Type: replace-cross Abstract: Long video summarization presents significant challenges for multimodal large language models (MLLMs), particularly in maintaining temporal fidelity over extended durations and producing summaries that are both semantically and temporally grounded.

By Alkesh Patel, Melis Ozyildirim, Ying-Chang Cheng, Ganesh Nagarajan
arXiv AI
Sep 10

TimeBlind: A Spatio-Temporal Compositionality Benchmark for Video LLMs

TimeBlind is a diagnostic benchmark designed to evaluate fine‑grained spatio‑temporal compositionality in video large language models (LLMs). It categorizes temporal understanding into three levels—atomic event recognition, event property characterization, and reasoning about event interdependencies—and uses a minimal‑pairs paradigm where video pairs share identical static content but differ only in temporal structure. Across 20 state‑of‑the‑art MLLMs tested on 600 curated instances, the best model achieved only 48.2% instance accuracy, far below human performance of 98.2%, highlighting a reliance on static visual shortcuts rather than true temporal reasoning.

By Baiqi Li, Kangyi Zhao, Ce Zhang, Chancharik Mitra, Jean de Dieu Nyandwi, Gedas Bertasius
arXiv Computer Vision
Oct 12

ELMA: Benchmarking Anaphoric Compositions for Long-Term Text-to-Motion Generation

The paper introduces ELMA, a new episodic long‑term motion dataset designed to benchmark anaphoric compositions in text‑to‑motion generation. ELMA contains densely annotated 30‑second clips extracted from in‑the‑wild videos, preserving cross‑segment dependencies that current models struggle to handle. Fine‑tuning state‑of‑the‑art models on ELMA improves both global physical realism and semantic alignment, establishing a robust baseline for long‑term motion synthesis.

By Shun Kato, Dixin Yang, Kotaro Amaya, Yuiga Wada, Kazumi Fukuda, Takashi Shibuya, Yuki Mitsufuji, Mariko Isogawa
arXiv AI
Jun 6

Seeing Time: Benchmarking Chronological Reasoning and Shortcut Biases in Vision-Language Models

arXiv:2606. 05702v1 Announce Type: new Abstract: Recent advancements in Vision-Language Models (VLMs) have significantly enhanced their ability to interpret complex visual semantics, yet their capacity for chronological reasoning remains under-explored.

By Haoyu Zhou, Qing Qing, Caichong Li, Qixin Zhang, Yongcheng Jing, Ziqi Xu, Juncheng Hu, Xikun Zhang, Renqiang Luo