The paper presents Real‑Time EXPO‑FT, a reinforcement learning framework that fine‑tunes large Vision‑Language‑Action models for real‑time robotic control. It separates slow, expressive action generation from fast, reactive edits, allowing a lightweight policy to adjust actions based on the latest observation. Experiments on the Kinetix benchmark and four dynamic real‑world tasks show that Real‑Time EXPO‑FT achieves superior performance, improving policy success rates from 42% to 97% with only ten minutes of online data and no human intervention.
By Perry Dong, Kuo-Han Hung, Dorsa Sadigh, Chelsea Finn
The paper introduces PARTS, a real‑world subtask reinforcement learning framework that fine‑tunes a pretrained robot policy by focusing on critical bottleneck subtasks while keeping the base policy frozen. It uses agent‑generated selectors and success verifiers to provide local rewards, enabling learning even when full‑task successes are rare. Experiments on bimanual YAM and single‑arm Franka robots show that PARTS raises complete‑task success from 32% to 61% and from 50% to 95%, respectively, with only tens of minutes of real‑world RL rollouts and minimal human intervention.
By Sichang Su, Benjamin Yang, Zhiyun Deng, Boyuan Liang, Yip Fun Yeung, Zelin Wang, Lingfeng Sun
arXiv:2605. 30226v2 Announce Type: replace-cross Abstract: Vision-Language-Action (VLA) models have emerged as a promising paradigm for grounding visual-language understanding into real-world robotic manipulation.
By Zhongxi Chen, Yifan Han, Yanming Shao, Huanming Liu, Congsheng Xu, Xiaoyu Chen, Yao Mu, Wenzhao Lian
arXiv:2606. 27475v1 Announce Type: cross Abstract: Robots trained on real world data tend to be imprecise, slow, and brittle to perturbations.
By Raymond Yu, William Huey, Mustafa Mukadam, Anusha Nagabandi, Abhishek Gupta
arXiv:2606. 12372v1 Announce Type: cross Abstract: Human-in-the-loop reinforcement learning (HiL-RL) has emerged as an effective paradigm for real-world robotic manipulation, enabling online policy improvement with human guidance.
By Haoyuan Deng, Yitong Gao, Yudong Lin, Haichao Liu, Zhenyu Wu, Ziwei Wang
EXPO-FT is a system that enables stable, sample‑efficient reinforcement learning fine‑tuning of pretrained Vision‑Language‑Action (VLA) policies. It achieves perfect success on a range of manipulation tasks—such as routing string lights, striking a pool ball, and inserting a flower into a wine bottle—using only about 19.1 minutes of online robot data. The approach outperforms both RL-from-scratch and existing VLA fine‑tuning methods, and the authors provide an open‑source codebase to support wider adoption.
By Perry Dong, Kuo-Han Hung, Tian Gao, Dorsa Sadigh, Chelsea Finn
The paper introduces QWM, a framework that integrates world models with standard Q‑learning to perform test‑time search over imagined trajectories. By training the policy and value function solely on real transitions, QWM avoids compounding model bias while still benefiting from predictive search. Experiments on the Robomimic and LIBERO manipulation benchmarks show that QWM outperforms strong prior state‑of‑the‑art methods in both sample efficiency and performance.
By Perry Dong, Yueru Jia, Chelsea Finn, Dorsa Sadigh
Prioritized Rollouts for Efficient World Model-based Vision-Language-Action Policy Optimization introduces U‑GROW, a lightweight sampling layer that directs more model rollouts toward states with high policy uncertainty, identified as decision‑sensitive stages where small action differences can alter task outcomes. By modifying only the branched‑start distribution, U‑GROW can be integrated into existing model‑based reinforcement learning pipelines without changing the policy optimization objective. Experiments on simulated and real‑world manipulation tasks demonstrate that U‑GROW improves the efficiency and effectiveness of policy optimization for Vision‑Language‑Action models.
By Yifei Sheng, Haoxiang Ren, Zhilong Zhang, Haonan Wang, Runjie Xu, Yihao Sun, Nan Tang, Zhichao Wu, Lei Yuan, Haoxin Lin, Yang Yu
arXiv:2609.36588v1 Announce Type: cross
Abstract: We study reinforcement learning (RL) methods for cooperative multi-agent Vision-Language-Action (VLA) models. This problem is challenging because VLA...
By Ruixiao Xu, Wong Lik Hang Kenny, Zhiqian Liu, Jianing Guo, Hanxiao Li, Kejian Shi, Shuning Zhang, Pu Feng, Yongjia Ma, Yuqing Ma, Kai Chen, Qi Dou, Yaodong Yang, Xianglong Liu, Simin Li
arXiv:2608. 19891v1 Announce Type: new Abstract: How to efficiently finetune robot policies to learn new tasks on the fly?
By Bhavya Sukhija, Oliver Groth, Mohit Shridhar, Tim Hertweck, Michael Bloesch, Markus Wulfmeier, Abbas Abdolmaleki, Martin Riedmiller
arXiv:2606. 31846v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models offer a promising framework for robotic manipulation by connecting language instructions, visual observations, and continuous control.
By Lang Cao, Renhong Chen, Luyi Li, Peng Wang, Mofan Peng, Yitong Li
arXiv:2604. 13733v2 Announce Type: replace-cross Abstract: Reinforcement learning (RL) enables high-frequency, closed-loop control for robotic manipulation, but scaling to long-horizon tasks with sparse or imperfect rewards remains difficult due to inefficient exploration and poor credit assignment.
By Angelo Moroncelli, Roberto Zanetti, Marco Maccarini, Loris Roveda