arXiv AI

When Should a VLM Look? Paying Only for Visual Calls That Were Needed and Used

arXiv AI
3d ago

Safety as a Budget: Calibrating the Control Channel of Constrained GRPO for Vision-Language Models

The paper proposes a two‑channel constrained GRPO framework that treats safety as explicit budgets on harmful assistance and over‑refusal rates, using Lagrange multipliers and a monotone mapping of scorer outputs. It demonstrates that this calibration reduces harmful responses from 8.9% to 4.8% on a Qwen3‑VL‑2B policy while keeping over‑refusal low, outperforming standard GRPO and uncalibrated budgets across multiple benchmarks.

By Bo Han
arXiv AI
Oct 12

Unsupervised Multimodal Reward Modeling

The paper presents an unsupervised multimodal reward model built on a frozen Qwen3‑VL backbone. By adding a linear head to an intermediate layer and training it to rank a web caption higher than a perturbed version of itself, the authors avoid human preference labels and achieve competitive performance—outperforming GPT‑4o on several benchmarks with far fewer parameters and compute. The approach demonstrates that a carefully chosen intermediate layer, simple perturbation-based training, and verifier‑driven fine‑tuning can yield effective vision‑language rewards without costly annotation.

By Yueying Li, Jingxuan Fan, Kiant\'e Brantley, Sham M. Kakade, Hanlin Zhang
arXiv AI
3d ago

CARE-PO: Controllable Tool-Call Efficiency for LLM Agents

The paper introduces CARE-PO, a method for controlling tool‑call efficiency in large language model agents. By anchoring an efficiency reward to a fixed reference budget tied to task difficulty and injecting successful trajectories into training rollouts, CARE‑PO prevents the collapse to zero tool calls and allows a single training run to produce multiple operating points balancing efficiency and capability. Experiments on the GAIA benchmark and other multi‑hop tasks show that CARE‑PO improves both success rates and call counts across several model sizes.

By Xiaoyang Ye, Xiao Yao, Hairui Zhu, Yiying Yang, Bei Gan, Jinliang Lu, Tengjin Weng, Yuan Yao, Wenhao Jiang
arXiv Machine Learning
Oct 3

Same Reward, Different Skills: When Multimodal RL Learns to Look

The paper demonstrates that reinforcement learning with verifiable rewards (RLVR) can improve vision‑language benchmark performance even when models are trained without visual input. When real images are introduced at test time, models trained blind recover about half of the performance gain at 3B parameters and nearly four‑fifths at 7B, but extended real‑image training can erode grounding while benchmark gains persist. The authors propose a visual resolvability rule and show that requiring visual evidence for correct answers leads to significant improvements in target discovery and generalization to unseen question types, while controls confirm that the gains stem from actual visual grounding rather than artifacts.

By Haocun Ye, Xinlong Jiang, Qile Chen, Bingyu Wang, Teng Zhang, Shubai Chen, Tingyu Wu, Zhenkun Zheng, Yiqiang Chen
arXiv Computer Vision
Oct 12

Auditing VLM Benchmarks from Existing Evaluation Runs

The paper introduces an audit layer for vision‑language model benchmarks that reuses per‑question artifacts from evaluation runs to assess the benchmark itself. Six detectors analyze saved outputs, and an additional text‑only rerun removes visual inputs to produce per‑question reports and aggregate statistics, including permutation‑null excess and interval estimates. Applying this method to the MMStar benchmark reveals that 20.2% of its questions show no accuracy gain from images, with a small but significant excess in mathematics and science categories, and identifies questions where all models chose the same non‑gold option.

By Denis Degtiarev, Anna Gromova, Sergey Katkov, Artem Mikhailin, Safina Mukhamedshina, Albert Nasybullin, Karina Shaikhutdinova, Artem Tuzov
arXiv Machine Learning
Oct 8

CARE: Certifying Acceleration for Vision-Language-Action Inference

The paper introduces CARE, a method for certifying accelerated inference in vision‑language‑action models. CARE uses paired rollouts on a calibration set to guarantee that the risk of acceleration‑induced failure stays below a user‑specified budget, selecting the fastest certified accelerator or falling back to the reference policy. Experiments on LIBERO suites show that CARE achieves 9.0–10.8× speedups while preserving at least 85.8% of reference‑solved episodes with 95% confidence, and it outperforms non‑guaranteed selectors in both budget adherence and rollout efficiency.

By Rui Liu, Tong Zheng, Jindong Gu, Zhipeng Wang
arXiv Computer Vision
Oct 28

How Much Can Best-of-N Help a World Action Model? Measuring Selectable Headroom and Realizing It in Closed Loop

The paper introduces a denoised counterfactual protocol to accurately measure the selectable headroom of world action models (WAMs) and uses these measurements to build a fail‑gated selector. Across four WAMs and two benchmarks, the study shows that the standard estimate overstates selectable headroom, that structured action perturbations can add up to 16 points of headroom, and that imagined futures retain candidate‑outcome information without loss. The selector improves closed‑loop task success from 76.9 % to 81.6 % on held‑out training episodes and from 62.6 % to 69.8 % on unseen tasks.

By Haojia Gao, Yue Cao, Yumin Liu, Haohua Que, Mingkai Liu, Guanqiu Guo, Zhuo Zou, Bokui Chen, Xueqian Wang