The paper investigates how reinforcement learning from verifiable rewards (RLVR) affects video‑language models’ understanding of time. Four open models were fine‑tuned under different data recipes—verified synthetic questions, unverified self‑supervised tasks, a 1:1 mix, and a verified+real mix—and evaluated on in‑domain and out‑of‑domain real‑video benchmarks with frames ordered, shuffled, or absent. Results show that verified training boosts in‑domain accuracy, largely through non‑visual gains, but can severely harm out‑of‑domain performance unless real verified data is added; moreover, no recipe induces genuine temporal‑order grounding, indicating that verifiable rewards improve benchmark scores without teaching temporal understanding.
By Avyay Sadhu, Patrick Cooper
The paper introduces Behavior Pack Optimization (BPO), a post‑training method for video multimodal large language models that replaces single‑response rewards with a set of outputs across counterfactual views. BPO enforces stability when interventions are irrelevant, sensitivity when key evidence is removed, and abstention when no evidence remains, using an anchor‑relative advantage to keep the objective stable with small pack sizes. Experiments on datasets such as TempCompass, MVBench, and NExT‑QA show that BPO improves macro accuracy and abstention metrics for models like Qwen2.5‑VL‑7B‑Instruct, with gains that transfer to other benchmarks and models.
By Zhaolu Kang, Shiyu Liu, Tailong Luo, Wei Zhang, Yingjie He, Lei Wei, Guansu Wang, Liang He, Siheng Wang, Guangyuan Dong, Jiaqi Su, Shuang Chen, Haoyu Ji, Qishi Zhan, Kaiyue Zhou
arXiv:2606. 01485v1 Announce Type: cross Abstract: We describe our submission to the VRR Challenge @ CVPR 2026, built on the \emph{ImplicitQA} / \emph{VRR-QA} benchmark~\cite{implicitqa}: multiple-choice video question answering in which answers are deliberately \emph{not} observable in any single frame and must be inferred from spatial layout, motion, depth, viewpoint, causality, and social context across discontinuous frames of creative video.
By Ali Alavi
arXiv:2609.09528v1 Announce Type: new
Abstract: Video large language models (Video-LLMs) are increasingly used as the perceptual front end of world models, a role that assumes they can read motion: h...
By Dhairya Bhatia, Bishoy Galoaa, Oliver Fritsche, Shahid Kamal, Muhammad Obaidullah Abdul Salam, Umer Saleem, Om Rastogi, Frania Felix Chettiar, Nesli Erdogmus, Sarah Ostadabbas
SCOUT is a frozen‑encoder approach for sim‑to‑real text‑based person retrieval that predicts cross‑modal embeddings instead of fine‑tuning cross‑encoders. It uses a trainable predictor to map patch tokens from a frozen video encoder (V‑JEPA) into the embedding space of a frozen text encoder (EmbeddingGemma), guided by a bidirectional InfoNCE objective. The method achieves state‑of‑the‑art results on the AI City Challenge 2026 Track 4, with a full retrieve‑fuse‑rerank pipeline reaching 84.25 mAP@10 and a single frozen model alone scoring 60.63, while training costs are modest (≈95 GPU‑hours).
By Abdarahmane Traor\'e, Andy Couturier, \'Eric Hervet
The paper introduces a training‑free method called “Track, Then Bind” for video‑language models to reason about object identity over time. It separates semantic identity binding from temporal identity maintenance by using a promptable video tracker to follow candidate objects and a frozen vision‑language model to bind an object to a track based on visible evidence. The approach achieves state‑of‑the‑art performance on several benchmarks, outperforming fine‑tuned trajectory models and other training‑free agents on tasks such as VET‑Bench, Perception Test cup games, and TOC‑Bench.
By Di Yang, Xiaochang Li, Muyang Liu, Yufei Wang, Yanhai Xiong
arXiv:2609.39563v1 Announce Type: new
Abstract: Existing video language models encode sampled RGB frames independently, so a long video must either exhaust the token budget or drop the changes betwee...
By Can Zhang, Xiaotian Han, Junyuan Shang, Yuchen Ding, Zhenyu Zhang, Shuohuan Wang, Dianhai Yu, Ruirui Li
The paper introduces GAVAGAI, a method that adds a referential alignment variable to vision‑language models by solving a small assignment problem via entropic optimal transport. By allowing words to be assigned to a special NOTHING column and enforcing a capacity constraint on regions, the approach improves word‑to‑picture accuracy and localization on the SAYCam infant video corpus. Experiments show that the alignment loss boosts performance over a standard next‑token captioning baseline, especially under high non‑reference rates.
By Rajdeep Pal, Jotiraditya Banerjee, Saptarshi Pani
arXiv:2606. 00910v1 Announce Type: cross Abstract: Composed Video Retrieval (CoVR) seeks the target video that results from applying a free-form textual modification to a reference video.
By Ali Alavi
arXiv:2605. 21854v2 Announce Type: replace-cross Abstract: Vision-Language-Action (VLA) models have rapidly converged on a small set of architectural patterns: discrete-token autoregression (e.
By Zhi Liu
The paper investigates using raw, unlabeled video as a mid‑training phase for a pretrained language model. By encoding video frames into visual tokens and training the model to predict the next token, the authors mid‑train Qwen3‑1.7B on YT‑Temporal‑1B clips. The resulting model outperforms a non‑mid‑trained counterpart by 2.9 points on video benchmarks and 5.1 points on image benchmarks, while maintaining comparable text performance.
By Jaedong Hwang, Xiaoqian Shen, Ernie Chang, Changsheng Zhao, Chong Zhou, Saksham Suri, Qi Qian, Zechun Liu, Lemeng Wu, Qinsi Wang, Raghuraman Krishnamoorthi, Wei Wen
The paper introduces a perception interface that separates vision from language in vision‑language models. A frozen perception stack detects objects, a deterministic semantic serializer converts the perceived state into text, and a standard text‑only large language model (LLM) answers questions. Experiments on a campus‑robot benchmark show that this serialized interface outperforms a zero‑shot VLM of the same language‑model size, especially as the language model shrinks, and that the advantage persists under paraphrase and different supervision regimes.
By Cong Xu, Ravi Sankar