← Back to all news
arXiv Machine Learning By Xinrui He, Ting-Wei Li, Junting Wang, Mengting Ai, Xinyu He, Hanghang Tong, Jingrui He

Harnessing Image Question Dependence for Better VLM Test-time Reinforcement Learning

Read the original on arXiv Machine Learning →

The paper introduces TTIQ, a test‑time reinforcement learning framework that improves vision‑language model (VLM) adaptation by explicitly measuring image‑question dependence. By teacher‑forcing responses on the original and ablated image‑question pairs, TTIQ derives token‑level likelihood changes to estimate how much each input contributes, then uses these signals to construct a reward that favors jointly grounded, confident responses. Experiments on eight VQA datasets and various VLM sizes show that TTIQ consistently outperforms prior consensus‑based methods and generalizes across model families and unseen datasets.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
3d ago

Stepwise Intrinsic Rewards for Reasoning in Large Language Models

The paper introduces Stepwise Marginal Information Gain (MIG), an intrinsic process reward that evaluates how each reasoning step of a large language model (LLM) or vision-language model (VLM) improves the likelihood of the reference answer. MIG rewards only new likelihood maxima, preventing duplicate credit, and is combined with outcome, format, and self‑distillation objectives to guide training. Experiments on eight benchmarks show that this method outperforms outcome‑only reinforcement learning and improves accuracy by up to 4.8 points over binary‑reward training, including a 12.6‑point gain on MathVerse and a 12.9‑point advantage on vision‑language transfer at 7B parameters.

By Xiangwei Wang, Wei Wang, Ken Chen, Nanduni Nimalsiri, Sachith Seneviratne, Saman Halgamuge
llmsragreinforcement-learningefficiencymultimodalbenchmarks
More like this →
arXiv Computer Vision
Sep 16

SAVOR: Self-Aware Visual Grounding via Confidence-Calibrated Reinforcement Learning for Multimodal Hallucination Mitigation

SAVOR is a training framework for multimodal large language models that adds token and answer confidence to the output schema, optimises a Group Relative Policy Optimisation objective to penalise calibration error and poor abstention, and uses the learned confidence at inference to revisit visual evidence only when uncertain. Experiments on POPE, HallusionBench, AMBER, and MMHal-Bench with InternVL3-8B and Qwen3-VL-8B backbones show that SAVOR reduces hallucination while maintaining general capability on MME and MMBench, achieving lower Expected Calibration Error than DPO and decoding baselines.

By Zixiu Ding, Zilin Zhao, Yingjie He, Xinlang Kang, Guansu Wang, Wei Zhang
llmsnlpreinforcement-learningmultimodalsafety
More like this →