arXiv AI By Keuntae Kim, Dongjun Kim, Yong Suk Choi

V-JEPA-LM: Video-Language Models That Read Dynamics Beyond Static, Text-Aligned Representations

Read the original on arXiv AI →

The paper introduces V-JEPA-LM, a video‑language model that leverages the dynamic representations learned by V‑JEPA video encoders. By applying counterfactual supervision—pixel transformations that create answer‑balanced video variants—the model learns to read motion dynamics such as speed, direction, and order, rather than merely retrieving static attributes. Trained on 4.53 M public samples with 32 visual tokens per frame, V‑JEPA‑LM outperforms existing open‑source video LLMs across eleven benchmarks, with a 4 B model achieving a 72.1 average score.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computer Vision
Oct 23

What Do Verifiable Rewards Teach Video-Language Models About Time? A Controlled Multi-Model Study

The paper investigates how reinforcement learning from verifiable rewards (RLVR) affects video‑language models’ understanding of time. Four open models were fine‑tuned under different data recipes—verified synthetic questions, unverified self‑supervised tasks, a 1:1 mix, and a verified+real mix—and evaluated on in‑domain and out‑of‑domain real‑video benchmarks with frames ordered, shuffled, or absent. Results show that verified training boosts in‑domain accuracy, largely through non‑visual gains, but can severely harm out‑of‑domain performance unless real verified data is added; moreover, no recipe induces genuine temporal‑order grounding, indicating that verifiable rewards improve benchmark scores without teaching temporal understanding.

By Avyay Sadhu, Patrick Cooper
arXiv Computer Vision
Oct 5

Behavior Pack Optimization for Video MLLM Post-Training

The paper introduces Behavior Pack Optimization (BPO), a post‑training method for video multimodal large language models that replaces single‑response rewards with a set of outputs across counterfactual views. BPO enforces stability when interventions are irrelevant, sensitivity when key evidence is removed, and abstention when no evidence remains, using an anchor‑relative advantage to keep the objective stable with small pack sizes. Experiments on datasets such as TempCompass, MVBench, and NExT‑QA show that BPO improves macro accuracy and abstention metrics for models like Qwen2.5‑VL‑7B‑Instruct, with gains that transfer to other benchmarks and models.

By Zhaolu Kang, Shiyu Liu, Tailong Luo, Wei Zhang, Yingjie He, Lei Wei, Guansu Wang, Liang He, Siheng Wang, Guangyuan Dong, Jiaqi Su, Shuang Chen, Haoyu Ji, Qishi Zhan, Kaiyue Zhou
arXiv Machine Learning
Jun 2

Perception First: A Frontier Native-Video Model with Self-Consistency for Implicit Video Question Answering

arXiv:2606. 01485v1 Announce Type: cross Abstract: We describe our submission to the VRR Challenge @ CVPR 2026, built on the \emph{ImplicitQA} / \emph{VRR-QA} benchmark~\cite{implicitqa}: multiple-choice video question answering in which answers are deliberately \emph{not} observable in any single frame and must be inferred from spatial layout, motion, depth, viewpoint, causality, and social context across discontinuous frames of creative video.

By Ali Alavi
arXiv Computer Vision
Sep 10

MotionBlind: Probing the Illusion of Motion Understanding in Video-LLMs

arXiv:2609.09528v1 Announce Type: new Abstract: Video large language models (Video-LLMs) are increasingly used as the perceptual front end of world models, a role that assumes they can read motion: h...

By Dhairya Bhatia, Bishoy Galoaa, Oliver Fritsche, Shahid Kamal, Muhammad Obaidullah Abdul Salam, Umer Saleem, Om Rastogi, Frania Felix Chettiar, Nesli Erdogmus, Sarah Ostadabbas
arXiv Computer Vision
Sep 18

SCOUT: Sim-to-Real Text-Based Person Retrieval by Embedding-Space Prediction over Frozen Video Features

SCOUT is a frozen‑encoder approach for sim‑to‑real text‑based person retrieval that predicts cross‑modal embeddings instead of fine‑tuning cross‑encoders. It uses a trainable predictor to map patch tokens from a frozen video encoder (V‑JEPA) into the embedding space of a frozen text encoder (EmbeddingGemma), guided by a bidirectional InfoNCE objective. The method achieves state‑of‑the‑art results on the AI City Challenge 2026 Track 4, with a full retrieve‑fuse‑rerank pipeline reaching 84.25 mAP@10 and a single frozen model alone scoring 60.63, while training costs are modest (≈95 GPU‑hours).

By Abdarahmane Traor\'e, Andy Couturier, \'Eric Hervet
arXiv Computer Vision
Oct 23

Ask About the Right Moment: Training-Free Object Identity Reasoning for Video-Language Models

The paper introduces a training‑free method called “Track, Then Bind” for video‑language models to reason about object identity over time. It separates semantic identity binding from temporal identity maintenance by using a promptable video tracker to follow candidate objects and a frozen vision‑language model to bind an object to a track based on visible evidence. The approach achieves state‑of‑the‑art performance on several benchmarks, outperforming fine‑tuned trajectory models and other training‑free agents on tasks such as VET‑Bench, Perception Test cup games, and TOC‑Bench.

By Di Yang, Xiaochang Li, Muyang Liu, Yufei Wang, Yanhai Xiong