arXiv AI

ReGRPO: Reflection-Augmented Policy Optimization for Tool-Using Agents

arXiv:2606. 31392v1 Announce Type: new Abstract: Tool-augmented vision-language models (VLMs) can solve multimodal, multi-step tasks by calling external tools, yet they remain fragile in practice.

arXiv AI

Jun 3

ToolGate: Token-Efficient Pre-Call Control for Tool-Augmented Vision-Language Agents

arXiv:2606. 03054v1 Announce Type: new Abstract: Tool-augmented vision-language agents can acquire external perceptual evidence through OCR, detection, segmentation, and other tools, but executing every proposed tool call is costly and sometimes unnecessary.

By Anjie Liu, Yan Song, Zhixun Chen, Ziqin Gong, Zhongwei Yu, Jun Wang

agents computer-vision multimodal benchmarks

arXiv AI

Jun 15

Closing the Reflection Gap: A Free Calibration Bonus for Agentic RL

arXiv:2606. 14211v1 Announce Type: new Abstract: LLMs are increasingly deployed as agents that interact with external environments and observe feedback such as execution results, error messages, and tool outputs.

By Yinglun Zhu

llms agents reinforcement-learning benchmarks data-engineering

arXiv AI

22h ago

EXPO-FT: Sample-Efficient Reinforcement Learning Finetuning for Vision-Language-Action Models

arXiv:2605. 25477v2 Announce Type: replace-cross Abstract: The ability to efficiently and reliably learn new tasks has been a foundational challenge in robotics.

By Perry Dong, Kuo-Han Hung, Tian Gao, Dorsa Sadigh, Chelsea Finn

reinforcement-learning robotics fine-tuning multimodal

arXiv AI

Jun 4

Can Generalist Agents Automate Data Curation?

arXiv:2606. 04261v1 Announce Type: new Abstract: Curating training data is among the most consequential yet labor-intensive parts of modern AI development: practitioners iteratively propose, implement, evaluate, and revise data policies against noisy benchmark feedback.

By Feiyang Kang, Hanze Li, Adam Nguyen, Mahavir Dabas, Jiaqi W. Ma, Frederic Sala, Dawn Song, Ruoxi Jia

llms agents multimodal benchmarks

arXiv Machine Learning

Jun 25

Why Multi-Step Tool-Use Reinforcement Learning Collapses and How Supervisory Signals Fix It

arXiv:2606. 26027v1 Announce Type: cross Abstract: Tool use enables large language models (LLMs) to perform complex tasks, and recent agentic reinforcement learning (RL) methods show promise for enhancing model capabilities.

By Yupu Hao, Zhuoran Jin, Huanxuan Liao, Kang Liu, Jun Zhao

llms agents reinforcement-learning fine-tuning

arXiv AI

Jun 30

Demonstration-Free Robotic Control via LLM Agents

arXiv:2601. 20334v2 Announce Type: replace-cross Abstract: Robotic manipulation has increasingly adopted vision-language-action (VLA) models, which achieve strong performance but typically require task-specific demonstrations and fine-tuning, and often generalize poorly under domain shift.

By Brian Y. Tsui, Alan Y. Fang, Tiffany J. Hwu

llms agents robotics fine-tuning multimodal benchmarks

arXiv AI

2d ago

Evolve Vision-Language-Action Model into an Agent with On-the-fly Tool-use

arXiv:2608. 14047v1 Announce Type: cross Abstract: This paper integrates end-to-end Visual-Language-Action (VLA) models with agentic tool-use to propose Agentic Robot with Tool-use (ART).

By Yi Ding, Yanzhao Yu, Xili Dai, Xianbiao Qi, Peiwen Sun, Xueqian Wang, Xiangyu Yue, Jianan Wang

agents robotics multimodal

arXiv Machine Learning

Jun 11

OGPO: Sample Efficient Full-Finetuning of Generative Control Policies

arXiv:2605. 03065v2 Announce Type: replace Abstract: Generative control policies (GCPs), such as diffusion- and flow-based control policies, have emerged as effective parameterizations for robot learning.

By Sarvesh Patil, Mitsuhiko Nakamoto, Manan Agarwal, Shashwat Saxena, Jesse Zhang, Giri Anantharaman, Cleah Winston, Chaoyi Pan, Douglas Chen, Nai-Chieh Huang, Zeynep Temel, Oliver Kroemer, Sergey Levine, Abhishek Gupta, Hongkai Dai, Paarth Shah, Max Simchowitz

diffusion reinforcement-learning robotics fine-tuning benchmarks

arXiv AI

Jun 4

Plan, Watch, Recover: A Benchmark and Architectures for Proactive Procedural Assistance

arXiv:2606. 04970v1 Announce Type: cross Abstract: We envision a proactive multi-modal assistant system which gives users real-time step-by-step guidance on a procedural task, autonomously deciding \textit{when} to interrupt, and \textit{how} to coach.

By Kaustav Kundu, Ritvik Shrivastava, Maxim Arap, Nanshu Wang, Xianhui Zhu, Quintin Fettes, Gautam Tiwari, Parth Suresh, Th\'eo Moutakanni, Alejandro Castillejo Munoz, Allen Bolourchi, Pascale Fung, Pinar Donmez, Babak Damavandi, Anuj Kumar, Seungwhan Moon

llms multimodal benchmarks

arXiv AI

Aug 10

AgentPatch: Coarse-to-Fine Weak-Task Repair for Merging Agentic Multimodal Large Language Models

arXiv:2608. 06699v1 Announce Type: new Abstract: Agentic multimodal large language models (MLLMs) extend multimodal perception and reasoning with planning, tool use, and interaction in dynamic environments.

By Zibo Shao, Baochen Xiong, Chengdong Xu, Linhui Xiao, Kaichen Li, Haoran Gong, Yan Li, Yaguang Song, Xiaoshan Yang

llms agents multimodal benchmarks

arXiv AI

Aug 6

RESample: A Robust Data Augmentation Framework via Exploratory Sampling for Robotic Manipulation

arXiv:2510. 17640v4 Announce Type: replace-cross Abstract: Vision-Language-Action (VLA) models have shown strong manipulation capability when trained with large-scale imitation learning datasets.

By Yuquan Xue, Guanxing Lu, Zhenyu Wu, Chuanrui Zhang, Bofang Jia, Zhengyi Gu, Ziwei Wang

robotics multimodal benchmarks

arXiv AI

Jul 1

Z-1: Efficient Reinforcement Learning for Vision-Language-Action Models

arXiv:2606. 31846v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models offer a promising framework for robotic manipulation by connecting language instructions, visual observations, and continuous control.

By Lang Cao, Renhong Chen, Luyi Li, Peng Wang, Mofan Peng, Yitong Li

reinforcement-learning robotics fine-tuning multimodal benchmarks

Related stories