arXiv Computer Vision

DexPolicy: Scheduled Exploration for Trajectory-Guided Dexterous Manipulation

arXiv Machine Learning
Jun 11

OGPO: Sample Efficient Full-Finetuning of Generative Control Policies

arXiv:2605. 03065v2 Announce Type: replace Abstract: Generative control policies (GCPs), such as diffusion- and flow-based control policies, have emerged as effective parameterizations for robot learning.

By Sarvesh Patil, Mitsuhiko Nakamoto, Manan Agarwal, Shashwat Saxena, Jesse Zhang, Giri Anantharaman, Cleah Winston, Chaoyi Pan, Douglas Chen, Nai-Chieh Huang, Zeynep Temel, Oliver Kroemer, Sergey Levine, Abhishek Gupta, Hongkai Dai, Paarth Shah, Max Simchowitz
arXiv Computer Vision
Sep 21

DexPIE: Stable Dexterous Policy Improvement from Real-World Experience

DexPIE is a post‑training framework that improves dexterous manipulation policies using real‑world experience. It introduces a dexterous‑hand‑adapted intervention system and multi‑stage DAgger‑style data collection to enhance exploration, aligns training and inference to reduce distribution shift, and conditions the policy on a continuous optimality indicator for fine‑grained data quality use. In three real‑world tasks, DexPIE boosts success rates by 37.3% over a demonstration‑based baseline, outperforming all other methods and showing stronger robustness.

By Ruizhe Liao, Wenrui Chen, Liangji Zeng, Haoran Lin, Fan Yang, Kailun Yang, Yaonan Wang
arXiv AI
Sep 21

SynthDemo-RL: Breaking the Zero-Reward Barrier in VLA Adaptation with LLM-Guided Synthetic Demonstrations

SynthDemo‑RL introduces a teacher‑student framework that uses an automated teacher to generate successful manipulation trajectories from simulator‑privileged state, which are then distilled into a Vision‑Language‑Action (VLA) student via supervised fine‑tuning. The student is further refined with PPO using binary task‑success rewards. On the LIBERO‑PRO benchmark, SynthDemo‑RL rescues all 27 previously unsolvable tasks and achieves near‑perfect success rates, matching performance that would otherwise require human demonstrations.

By Hiroaki Kingetsu, Hiroaki Kurihara, Kaoru Yokoo, Kenji Fukumizu, Manohar Kaul
arXiv Machine Learning
Sep 25

Uncertainty-Gated Exploration Noise Suppresses Task Collapse in Online RL Fine-Tuning of a Flow-Matching Vision-Language-Action Policy

The paper investigates task collapse—a failure mode where online RL fine‑tuning of a pretrained flow‑matching vision‑language‑action policy erodes performance on individual tasks—using a 450M‑parameter SmolVLA policy on LIBERO‑10. Three exploration‑noise strategies are compared: a fixed noise scale, a learned noise network, and an uncertainty‑gated controller that reallocates exploration based on novelty and competence signals without task labels. The uncertainty‑gated controller prevents task collapse across all tested seeds, whereas the other two approaches consistently cause collapse, demonstrating its effectiveness in preserving task performance during fine‑tuning.

By Mehmet Turan Yard{\i}mc{\i}, Yunus Emre \c{C}o\u{g}urcu
arXiv Machine Learning
Jul 21

Foresight Residual RL for Long-Horizon Robot Manipulation with Vision-Language-Action Models

arXiv:2607. 16506v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) policies offer strong general-purpose manipulation priors, but often fail on tight-tolerance, contact-rich assembly due to long-horizon credit assignment and subtask coupling: a state that is geometrically successful for the current skill can be brittle for downstream skills.

By Yuhan Liu, Xinyu Zhang, Litao Liu, Abdeslam Boularias
arXiv Machine Learning
Jul 31

REFINE-DP: Diffusion Policy Fine-tuning for Humanoid Loco-manipulation via Reinforcement Learning

arXiv:2603. 13707v3 Announce Type: replace-cross Abstract: Humanoid loco-manipulation requires coordinated task-space motion planning with stable loco-manipulation command tracking under complex robot-environment dynamics and long-horizon tasks.

By Zhaoyuan Gu, Yipu Chen, Zimeng Chai, Alfred Cueva, Thong Nguyen, Yifan Wu, Huishu Xue, Minji Kim, Isaac Legene, Fukang Liu, KyoungMok Kim, Ayan Barula, Yongxin Chen, Ye Zhao
arXiv Machine Learning
Aug 11

Directional-Clamp PPO

arXiv:2511. 02577v2 Announce Type: replace Abstract: Proximal Policy Optimization (PPO) is widely regarded as one of the most successful deep reinforcement learning algorithms, known for its robustness and effectiveness across a range of problems.

By Gilad Karpel, Ruida Zhou, Shoham Sabach, Mohammad Ghavamzadeh