arXiv Computer Vision

Behavior Pack Optimization for Video MLLM Post-Training

The paper introduces Behavior Pack Optimization (BPO), a post‑training method for video multimodal large language models that replaces single‑response rewards with a set of outputs across counterfactual views. BPO enforces stability when interventions are irrelevant, sensitivity when key evidence is removed, and abstention when no evidence remains, using an anchor‑relative advantage to keep the objective stable with small pack sizes. Experiments on datasets such as TempCompass, MVBench, and NExT‑QA show that BPO improves macro accuracy and abstention metrics for models like Qwen2.5‑VL‑7B‑Instruct, with gains that transfer to other benchmarks and models.

arXiv Computer Vision

Aug 24

Annotations as Rollouts: Efficient and Scalable Reinforcement Learning for Video MLLMs

The paper introduces OraRL, a reinforcement learning framework that leverages annotations as oracle rollouts to improve sample efficiency and scalability for video multimodal large language models (MLLMs). By decoupling advantage estimation and employing sign‑balanced pruning, OraRL achieves faster training and better performance across multiple video‑perception benchmarks compared to existing methods. The approach scales from 0.8B to 9B parameters and handles up to 100k prompts, delivering significant gains in temporal mIoU, tracking accuracy, segmentation, and spatial‑intelligence metrics.

By Yunheng Li, Guohong Mu, Hao Li, Shengsheng Qian, Dingwen Zhang, Qibin Hou, Ming-Ming Cheng

llms computer-vision reinforcement-learning fine-tuning efficiency multimodal benchmarks

arXiv Computer Vision

5d ago

Token-Level Video Reinforcement Learning

arXiv:2610.01973v1 Announce Type: new Abstract: Reinforcement learning (RL) for video generation usually assigns one scalar reward to an entire sampled video. Yet a video is not uniformly flawed: som...

By Yifan Wang, Gordon Guocheng Qian, Yanyu Li, Anil Kag, Yun Fu

llms diffusion reinforcement-learning multimodal

arXiv AI

Jun 24

video-SALMONN-R$^3$: Learning to ReWatch, ReAsk, and ReAnswer for Efficient Video Understanding

arXiv:2606. 24477v1 Announce Type: cross Abstract: Video large language models (LLMs) are often constrained by computation and memory budgets, leading them to use reduced frame rates and spatial resolutions, which may cause them to miss critical information for question answering (QA).

By Yixuan Li, Guangzhi Sun, Yudong Yang, Wei Li, Zejun MA, Chao Zhang

llms nlp reinforcement-learning fine-tuning

arXiv AI

Jun 11

ProcessThinker: Enhancing Multi-modal Large Language Models Reasoning via Rollout-based Process Reward

arXiv:2606. 11209v1 Announce Type: cross Abstract: Visual question answering increasingly requires multi-step reasoning.

By Jingpei Wu, Xiao Han, Weixiang Shen, Boer Zhang, Zifeng Ding, Volker Tresp

llms nlp reinforcement-learning fine-tuning multimodal benchmarks

arXiv Computer Vision

Aug 24

Enhancing Localized Reasoning for Long Video Understanding via Efficient Segment-to-Video Supervision

arXiv:2608.20814v1 Announce Type: new Abstract: Though Multimodal Large Language Models (MLLMs) have shown impressive potential in video understanding, long video understanding (LVU) remains challeng...

By Beibei Zhang, Chao Xu, Jun Lan, Zongyi Li, Lai Wei, Huijia Zhu, Tongwei Ren

llms reinforcement-learning fine-tuning efficiency multimodal benchmarks

arXiv Computation and Language

Sep 1

Multi-Agent Self-Improving Reinforcement Learning for Video Reasoning

arXiv:2608.28675v1 Announce Type: cross Abstract: Video reasoning tasks such as grounded video question answering and temporal grounding require selecting temporal evidence that supports the query. I...

By Mingwen Zhang, Jisheng Dang, Minqiang Yang, Bimei Wang, Bin Hu, Tat-Seng Chua

agents nlp reinforcement-learning fine-tuning benchmarks

arXiv AI

Sep 28

Stepwise Intrinsic Rewards for Reasoning in Large Language Models

The paper introduces Stepwise Marginal Information Gain (MIG), an intrinsic process reward that evaluates how each reasoning step of a large language model (LLM) or vision-language model (VLM) improves the likelihood of the reference answer. MIG rewards only new likelihood maxima, preventing duplicate credit, and is combined with outcome, format, and self‑distillation objectives to guide training. Experiments on eight benchmarks show that this method outperforms outcome‑only reinforcement learning and improves accuracy by up to 4.8 points over binary‑reward training, including a 12.6‑point gain on MathVerse and a 12.9‑point advantage on vision‑language transfer at 7B parameters.

By Xiangwei Wang, Wei Wang, Ken Chen, Nanduni Nimalsiri, Sachith Seneviratne, Saman Halgamuge

llms rag reinforcement-learning efficiency multimodal benchmarks

arXiv AI

Jun 8

Watch, Remember, Reason: Human-View Video Understanding with MLLMs

arXiv:2606. 07433v1 Announce Type: cross Abstract: Video understanding is being rapidly transformed by multimodal large language models (MLLMs), as research moves from short clips to long, multimodal, and knowledge-intensive video scenarios.

By Jiahao Meng, Yue Tan, Qi Xu, Kuan Gao, Weisong Liu, Yanwei Li, Jason Li, Lingdong Kong, Haochen Wang, Qianyu Zhou, Jiangning Zhang, Guangliang Cheng, Yunhai Tong, Lu Qi, Minghsuan Yang

llms multimodal benchmarks safety

arXiv Computer Vision

Sep 10

Beyond Frame Selection: Rethinking Long-Video Understanding with MLLMs

arXiv:2608.05592v2 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) have made strong progress in video understanding, yet long videos remain difficult: the visual token budge...

By Ziling Huang, Shin'ichi Satoh

llms multimodal benchmarks

arXiv Computer Vision

6d ago

RESUME: Recurrent State Updates from Motion and Residual Signals for Efficient Video Language Modeling

arXiv:2609.39563v1 Announce Type: new Abstract: Existing video language models encode sampled RGB frames independently, so a long video must either exhaust the token budget or drop the changes betwee...

By Can Zhang, Xiaotian Han, Junyuan Shang, Yuchen Ding, Zhenyu Zhang, Shuohuan Wang, Dianhai Yu, Ruirui Li

llms benchmarks

arXiv Machine Learning

Jul 7

Incentivizing Vision Language Models to Search for Long Video Question Answering

arXiv:2607. 02959v1 Announce Type: cross Abstract: We introduce VSeek, an agentic framework that transforms long-video question answering (LVQA) from a passive, single-pass perception task into a multi-turn retrieval process.

By Harsh Goel, S P Sharan, Sahil Shah, Minkyu Choi, Joungbin An, Kristen Grauman, Sandeep P. Chinchali

llms agents nlp reinforcement-learning benchmarks

arXiv AI

22h ago

Beyond Scalar IoU: Structured Verification from Rollout Groups for Video Temporal Grounding

The paper introduces SUTURE, a structured verification method for video temporal grounding that leverages the joint structure of rollout groups rather than scoring each rollout independently. SUTURE conditions verification on the entire rollout group, using disagreement across rollouts to reweight targets and coverage at each position to redistribute reward mass. Experiments on five benchmarks show that SUTURE consistently improves grounding performance across all IoU thresholds and reduces video-start anchoring in reasoning traces, indicating that the joint structure of a rollout group can provide a more informative temporal verifier.

By Youngjae Cho, Won Young Jhoo, Jongsuk Kim

reinforcement-learning benchmarks

Related stories