arXiv AI
CriticHack: Evaluating Visual Rewards Under Robot Policy Optimization
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Sep 25
Uncertainty-Gated Exploration Noise Suppresses Task Collapse in Online RL Fine-Tuning of a Flow-Matching Vision-Language-Action Policy
The paper investigates task collapse—a failure mode where online RL fine‑tuning of a pretrained flow‑matching vision‑language‑action policy erodes performance on individual tasks—using a 450M‑parameter SmolVLA policy on LIBERO‑10. Three exploration‑noise strategies are compared: a fixed noise scale, a learned noise network, and an uncertainty‑gated controller that reallocates exploration based on novelty and competence signals without task labels. The uncertainty‑gated controller prevents task collapse across all tested seeds, whereas the other two approaches consistently cause collapse, demonstrating its effectiveness in preserving task performance during fine‑tuning.
arXiv Machine Learning
Jul 13
SafeExplorer: An Unbiased Policy Gradient for Reinforcement Learning with Recovery Interventions
arXiv:2607. 08925v1 Announce Type: new Abstract: Training reinforcement-learning agents directly on physical robots makes every fall costly, since a fall can damage the platform and cannot be undone like a simulator reset; the goal is therefore to minimize falls during training rather than trade them off against return, as constrained Markov decision process (MDP) formulations do.
arXiv AI
Aug 5
Continue or Replan? Bernoulli-Continuation Policy Learning for Adaptive Horizon Execution
arXiv:2608. 03483v1 Announce Type: cross Abstract: Existing chunk-based Vision-Language-Action (VLA) models execute a fixed number of actions (i.
arXiv Machine Learning
Aug 19
Prism-GRPO: Faster VLA Policy Optimization via Splitting Same-outcome Groups
Prism‑GRPO enhances the GRPO reinforcement‑learning algorithm for vision‑language‑action policies by adding a weighted trajectory‑level execution‑quality score to binary success rewards. This approach splits groups with identical outcomes into a quality spectrum, preserving training signal while ensuring successes always outrank failures. Experiments on four RoboTwin tasks show Prism‑GRPO achieves higher success and quality at matched rollout budgets, reaching target success rates with up to 56% fewer rollouts and mitigating reward‑hacking behaviors that transfer to real‑robot deployment.
arXiv Machine Learning
Jul 13
Learning More from Less: Reinforcement Learning from Hindsight
arXiv:2607. 09042v1 Announce Type: new Abstract: Reinforcement learning (RL) is increasingly used to post-train vision-language-action (VLA) models, but every update consumes robot rollouts that are slow and costly to collect, making sample efficiency a central concern.
arXiv AI
Aug 18
Robo-Dopamine 2.0: History-Conditioned and OOD-Aware Process Reward Modeling for Robotic Manipulation
arXiv:2608. 15680v1 Announce Type: cross Abstract: Vision-language-action (VLA) models improve robotic manipulation but remain vulnerable to compounding errors, scene changes, and off-trajectory states.