REVERSAL-BENCH is a benchmark that introduces a continuous reversibility parameter ρ∈[0,1] and a reset oracle to evaluate how well reinforcement learning agents can recover from irreversible states across eight manipulation tasks in five physics engines. Experiments show a sharp reversibility cliff: reset‑free agents become trapped in irrecoverable states as ρ increases, while episodic agents continue learning steadily. The benchmark also provides a large multi‑simulator dataset and demonstrates that safety shields can predict recoverability but only succeed when the agent can avoid the trap.
arXiv:2606. 12372v1 Announce Type: cross Abstract: Human-in-the-loop reinforcement learning (HiL-RL) has emerged as an effective paradigm for real-world robotic manipulation, enabling online policy improvement with human guidance.
arXiv:2603. 16842v2 Announce Type: replace Abstract: Stochastic resetting -- intermittently returning a process to a fixed reference state -- has emerged as an effective mechanism for optimizing first-passage properties.
arXiv:2606. 15333v1 Announce Type: cross Abstract: LLM unlearning has emerged as a cost-effective alternative to full retraining for removing hazardous knowledge from pretrained models while preserving general utility.
arXiv:2609.38955v1 Announce Type: cross Abstract: Inverse Reinforcement Learning (IRL) aims to recover a reward function that explains expert demonstrations. Existing IRL methods typically rely on a...
arXiv:2608. 11977v1 Announce Type: new Abstract: Tool-using LLM agents are commonly trained and evaluated in environments where tool calls succeed reliably, yet deployed tools can fail transiently, persistently, or silently.
The paper introduces HALTER, a graph-based system that automates the reset and evaluation of long-horizon robot manipulation tasks. HALTER constructs a spatial scene graph from point clouds and vision models, uses an LLM to score rollouts, plan resets, and verify success, all without labeled success images. In experiments on a Franka arm, HALTER restores scenes in 76% of episodes, improves skill completion estimation, and reduces operator time by 72% compared to manual reset.
MAGMA-GEN is an on‑policy data‑generation pipeline that transforms ambiguous failures in hierarchical robotic manipulation into validated recovery supervision. It uses a privileged coach to hypothesize early decision‑level errors and proposes localized corrections, then retains only those candidates that improve downstream progress when re‑executed from the same state. This approach generates supervised examples from the agent’s own failure distribution, enabling improved task success and recovery without requiring per‑step human demonstrations.
arXiv:2607. 01111v1 Announce Type: cross Abstract: Robot policies inevitably encounter failures when deployed in real environments.
The paper introduces Reverse‑Turn Policy Optimization (RTPO), a method that restructures multi‑turn agentic reinforcement learning rollouts into sparse reverse trees and updates policies in temporal reverse order. This approach addresses three key instability sources—context mismatch, weak turn‑level credit assignment, and asynchronous policy drift—by aligning each decision with its downstream continuation. Theoretical analysis shows RTPO eliminates context mismatch and drift, reduces credit bias, and converges to recursive optimality, while experiments demonstrate performance gains of 21.50% over trajectory‑level and 10.76% over turn‑level baselines on multi‑turn agentic RL benchmarks.
arXiv:2604.16683v2 Announce Type: replace-cross Abstract: Imitation learning has enabled robots to acquire complex visuomotor manipulation skills from demonstrations, but deployment failures remain a...
The paper introduces On‑Policy Warmup (OPW), a teacher‑guided training stage where a student agent learns from a teacher on its own interaction trajectories before switching to reinforcement learning with verifiable rewards (RLVR). OPW differs from traditional imitation by focusing on states generated by the student’s own decisions, including imperfect actions and recovery situations. The authors provide a theoretical link between on‑policy reverse‑KL distillation and trajectory‑level distribution matching, showing that, under a competent teacher and low distillation loss, OPW can lower bound initial verifier success and reduce reward‑discovery complexity, thereby accelerating RLVR performance.
arXiv:2609.33875v2 Announce Type: replace-cross Abstract: Outcome-only reinforcement learning gives software engineering (SWE) agents a terminal success signal but little direct guidance about interm...