arXiv Machine Learning June 11, 2026 By Yifan Yang, Zhen Zhang, Jiayi Tian, Liyan Tan, Zheng Zhang

IAPO: Input Attribution-Aware Policy Optimization for Tool Use in Small Multimodal Agents

Read the original on arXiv Machine Learning →

arXiv:2606. 11652v1 Announce Type: new Abstract: This paper investigates reinforcement learning (RL) methods for improving tool-calling capabilities in multimodal small language model (SLM) agents.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv AI

Aug 3

Multimodal Reinforcement Learning with Adaptive Verifier for AI Agents

arXiv:2512. 03438v3 Announce Type: replace Abstract: Agentic reasoning models trained with multimodal reinforcement learning (MMRL) have become increasingly capable, yet they are almost universally optimized using sparse, outcome-based rewards computed based on the final answers.

By Reuben Tan, Baolin Peng, Zhengyuan Yang, Hao Cheng, Oier Mees, Theodore Zhao, Andrea Tupini, Isar Meijer, Qianhui Wu, Yuncong Yang, Lars Liden, Yu Gu, Sheng Zhang, Xiaodong Liu, Lijuan Wang, Marc Pollefeys, Yong Jae Lee, Jianfeng Gao

agents reinforcement-learning robotics fine-tuning multimodal benchmarks safety

arXiv AI

Jul 13

Task-Specific Multimodal Question Answering Agents via Confidence Calibration and Incremental Reasoning for QANTA 2026

arXiv:2607. 09623v1 Announce Type: cross Abstract: We present our submission to the QANTA 2026 shared challenge at the ICML 2026 Workshop on Efficient Multimodal Question Answering (EMM-QA).

By Nirjhar Das, Md. Al-Mamun Provath

llms agents nlp multimodal benchmarks

arXiv AI

Jul 13

Multimodal Reward Hacking in Reinforcement Learning

arXiv:2607. 09492v1 Announce Type: new Abstract: Reinforcement learning (RL) is increasingly used to align multimodal large language models (MLLMs), but higher rewards do not always imply better task performance.

By Jiayu Yao, Yiwei Wang, Anmeng Zhang, Zhe Sun, Songsong Wang, Lingrui Mei, Yuyao Ge, Shenghua Liu

llms reinforcement-learning fine-tuning multimodal safety

arXiv AI

Jun 30

Be Faithful When Response: Returning Fluent and Grounded Answers for Vision-Language Models Reinforcement Learning

arXiv:2606. 29984v1 Announce Type: new Abstract: Reinforcement Learning (RL) is an important paradigm for improving the reasoning capabilities of Vision-Language Models (VLMs).

By Peng, Lee, Yin Zhang, Yanglin Zhang, Haonan Wu, Zishan Liu, Ruoxi Zang, Xin Zhu, Jiayin Zheng, Jian Yao, Zefeng Ji, Fei Ma

llms reinforcement-learning multimodal benchmarks

arXiv AI

Jun 2

Do Multimodal Agents Really Benefit from Tool Use? A Systematic Study of Capability Gains

arXiv:2606. 02357v1 Announce Type: cross Abstract: Tool-augmented multimodal agents show strong benchmark gains, often taken as evidence that agents have learned to use tools.

By Garvin Guo, Donglei Yu, Yu Chen, Xiang Wang, Shuai Li, Xinpei Zhao, Huaxing Liu, Qinghao Wang, Minpeng Liao

agents computer-vision multimodal benchmarks

arXiv AI

Jun 11

ASRU: Activation Steering Meets Reinforcement Unlearning for Multimodal Large Language Models

arXiv:2605. 15687v2 Announce Type: replace-cross Abstract: Multimodal large language models (MLLMs) may memorize sensitive cross-modal information during pretraining, making machine unlearning (MU) crucial.

By Jiahui Guang, Haiyan Wang, Yingjie Zhu, Cuiyun Gao, Jing Li, Di Shao, Zhaoquan Gu

llms multimodal safety

Related stories

Multimodal Reinforcement Learning with Adaptive Verifier for AI Agents

Task-Specific Multimodal Question Answering Agents via Confidence Calibration and Incremental Reasoning for QANTA 2026

Multimodal Reward Hacking in Reinforcement Learning

Be Faithful When Response: Returning Fluent and Grounded Answers for Vision-Language Models Reinforcement Learning

Do Multimodal Agents Really Benefit from Tool Use? A Systematic Study of Capability Gains

ASRU: Activation Steering Meets Reinforcement Unlearning for Multimodal Large Language Models