arXiv Machine Learning
Jun 5
Unraveling the Hidden Dynamical Structure in Recurrent Neural Policies
arXiv:2602. 01196v2 Announce Type: replace Abstract: Recurrent neural policies are widely used in partially observable control and meta-RL tasks.
Robust Recurrent Reinforcement Learning under Evolving Hidden Disturbances with Application to Rover Wheel Slip
The paper studies recurrent Twin Delayed Deep Deterministic Policy Gradient (TD3) agents in environments with evolving hidden disturbances, focusing on how observation history, action history, history length, and network structure influence performance. Three recurrent architectures are compared under controlled disturbances, revealing that action history is crucial when responses depend on prior actions and that a unified temporal sequence of action-observation pairs outperforms separate branches. The authors introduce H‑TD3, which reuses actor-generated recurrent states to initialize the critic, and demonstrate that these architectures excel in a rover wheel‑slip simulation, with policies trained on temporally structured disturbances transferring better to unseen slip dynamics.
Related stories
arXiv Machine Learning
Jul 9
Latent Policy Steering through One-Step Flow Policies
arXiv:2603. 05296v2 Announce Type: replace-cross Abstract: Offline reinforcement learning (RL) allows robots to learn from offline datasets without risky exploration.
arXiv AI
2d ago
DriftOPD: Sequence-Level Reverse-KL Distillation for One-Step VLA Policies
DriftOPD is a teacher‑free, rollout‑free framework that performs sequence‑level on‑policy distillation of continuous Vision‑Language‑Action (VLA) action experts. It decomposes the sequence‑level reverse‑KL divergence into a chunk‑level reverse‑KL term and a future‑potential term, optimizing them with a one‑step drifting objective and a Q‑function critic learned from offline demonstrations. Experiments on multiple VLA architectures in simulation and real‑world manipulation show that DriftOPD outperforms existing one‑step distillation baselines while matching the task success of multi‑step teacher policies.
arXiv Computer Vision
Sep 18
Learning Foresight without Explicit Trajectories for 3D Diffusion Policies
The paper introduces Movement Trend Guidance, a method that equips 3D diffusion policies with foresight by learning a compact latent representation of interaction evolution from a brief observation history. This latent, supervised by sparse future gripper states during training, serves as future-oriented conditioning during inference, enhancing action generation without adding explicit planning. The approach improves performance on RoboTwin2.0, LIBERO-40, and DexArt benchmarks, achieving higher success rates across multiple tasks.
arXiv AI
Jul 31
DLAM: Distributional Latent Actions with Temporal Constraints
arXiv:2607. 27138v1 Announce Type: cross Abstract: Vision-language-action (VLA) models remain constrained by scarce action-labeled robot data, whereas action-free videos offer abundant observations of physical change.
arXiv Machine Learning
Jun 11
UniIntervene: Agentic Intervention for Efficient Real-World Reinforcement Learning
arXiv:2606. 12372v1 Announce Type: cross Abstract: Human-in-the-loop reinforcement learning (HiL-RL) has emerged as an effective paradigm for real-world robotic manipulation, enabling online policy improvement with human guidance.
arXiv AI
Jul 24
VPWEM: Non-Markovian Visuomotor Policy with Working and Episodic Memory
arXiv:2603. 04910v2 Announce Type: replace-cross Abstract: Imitation learning from human demonstrations has achieved significant success in robotic control, yet most visuomotor policies still condition on single-step observations or short-context histories, making them struggle with non-Markovian tasks that require long-term memory.
arXiv AI
Jun 11
Implicit Neural Representations of Individual Behavior
arXiv:2606. 12200v1 Announce Type: cross Abstract: We study policy representation learning from unlabeled multi-policy behavioral data.
arXiv Machine Learning
Aug 26
Learning to Act While Waiting: RL Finetuning of Generalist Robot Policies Under Inference Latency
The paper introduces ARLI, a latency‑aware framework that enables reinforcement learning fine‑tuning of large generalist robot policies despite inference delays. ARLI combines asynchronous inference with state augmentations—incorporating committed actions and mid‑inference observations—to restore near‑Markovian dynamics and maintain reactivity. Experiments on simulated and real‑world manipulation tasks show that ARLI allows effective policy improvement under latency, outperforming standard RL even in no‑latency scenarios.
arXiv Machine Learning
Aug 11
LUCID: Latent-Skill Unified Control via Imagined Dynamics for Long-Horizon Humanoid Loco-Manipulation
arXiv:2608. 07746v1 Announce Type: new Abstract: Long-horizon humanoid loco-manipulation requires composing versatile whole-body skills and reliable high-level decision making.
arXiv AI
4d ago
DRAM: Delta-rule Recurrent Associative Memory for Robot Manipulation Policies
arXiv:2609.32453v2 Announce Type: replace-cross Abstract: Robotic manipulation is inherently history-dependent, yet most pretrained robotic policies condition on only the current observation or a sho...
arXiv Machine Learning
Sep 21
Trajectory Entropy Reinforcement Learning for Robust Robot Motor Skill Learning
arXiv:2505.04193v2 Announce Type: replace Abstract: Simplicity is a critical inductive bias for designing data-driven controllers, especially when robustness is important. Despite the impressive resu...