arXiv Machine Learning
COMET: Contrastive Motion-Enhanced Temporal Reasoning for Video Multimodal Large Language Models
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Sep 10
TimeBlind: A Spatio-Temporal Compositionality Benchmark for Video LLMs
TimeBlind is a diagnostic benchmark designed to evaluate fine‑grained spatio‑temporal compositionality in video large language models (LLMs). It categorizes temporal understanding into three levels—atomic event recognition, event property characterization, and reasoning about event interdependencies—and uses a minimal‑pairs paradigm where video pairs share identical static content but differ only in temporal structure. Across 20 state‑of‑the‑art MLLMs tested on 600 curated instances, the best model achieved only 48.2% instance accuracy, far below human performance of 98.2%, highlighting a reliance on static visual shortcuts rather than true temporal reasoning.
arXiv Computer Vision
Sep 3
TAME: Temporal-Aware Mixture-of-Experts for Text-Video Retrieval
TAME is a CLIP‑based framework for Text‑Video Retrieval that incorporates temporal modeling through three key innovations: sparse Mixture‑of‑Experts layers with frame‑consistent routing, Frame‑Temporal tokens that aggregate cross‑frame information, and a Cross‑Temporal Interaction and Aggregation module for refining frame‑wise similarities. These components enable the model to capture both local visual patterns and long‑range temporal dependencies, leading to consistent performance gains over CLIP‑based baselines on multiple TVR benchmarks, including a 4.0 R@1 improvement on MSR‑VTT. The code is publicly available on GitHub.
Hugging Face Trending Papers
Sep 2
TAME: Temporal-Aware Mixture-of-Experts for Text-Video Retrieval
TAME introduces a Temporal-Aware Mixture-of-Experts framework for Text-Video Retrieval that enhances CLIP-based models by incorporating frame-level structure and temporal relations. It adds sparse Mixture-of-Experts layers with frame-consistent routing, Frame-Temporal tokens for global cross-frame aggregation, and a Cross-Temporal Interaction and Aggregation module to refine sentence-video similarities. Experiments on multiple TVR benchmarks show consistent performance gains, such as a 4.0 R@1 improvement on MSR‑VTT over CLIP4Clip.
arXiv Computation and Language
4d ago
FastBench: Can Streaming VLMs Perceive High-Dynamic Real-World Streams?
FastBench is a new benchmark that tests streaming Video‑Large‑Language Models (VLMs) on high‑dynamic real‑world video streams, where events occur rapidly and cannot be captured by sparse 1–2 FPS sampling. The benchmark uses a trajectory‑grounded pipeline that generates QA pairs from high‑FPS clips, filters questions answerable at 2 FPS, verifies answers with SAM3 and CoTracker3 trajectories, and incorporates three rounds of human inspection, yielding 306 QA pairs across eight domains and multiple temporal scopes. Experiments show that even the strongest model, Gemini‑3.5‑Flash, achieves only 50.7 % accuracy, and that denser sampling improves performance but saturates when history is compressed; a training‑free baseline, ProactiveFrame, outperforms uniform sampling but still falls short of oracle‑guided focusing.
arXiv AI
Jun 8
MotionEnhancer: Leveraging Video Diffusion for Motion-Enhanced Vision-Language Models
arXiv:2606. 06853v1 Announce Type: cross Abstract: The new era has witnessed a remarkable capability to extend Vision-Language Models (VLMs) for tackling tasks of video understanding.
arXiv AI
Jul 29
Reasoning with Memory: A Temporal Granularity-Adaptive Framework for Training-Free Long Video Understanding
arXiv:2607. 24794v1 Announce Type: new Abstract: While Multimodal Large Language Models (MLLMs) demonstrate superior generalization in fundamental video tasks, restricted context windows limit their long video understanding.