arXiv Machine Learning

CoDrift: Compositional Drifting for Offline Reinforcement Learning

arXiv Machine Learning
Sep 3

Recursive Value Learning for Long-Horizon Offline Goal-Conditioned RL

The paper introduces DCRL (Divide-and-Conquer RL), a method that recursively decomposes offline goal-conditioned reinforcement learning trajectories into a balanced binary tree. By training values from the leaves up to the root, DCRL avoids noisy max-based backups and reduces bootstrap depth from linear to logarithmic, thereby limiting error accumulation. Experiments on diverse goal-reaching tasks show that DCRL outperforms prior flat offline GCRL methods, achieving a higher average score on the most challenging long-horizon OGBench tasks.

By Hyeonseong Jeon, Youngwoon Lee
arXiv AI
Oct 9

Higher-Order Action Supervision Makes A Strong Policy Class

The paper proposes a new loss scheme that simultaneously supervises zeroth- and first-order actions to improve policy performance and control robustness in data-driven decision-making methods such as imitation learning and reinforcement learning. The approach provides formal theoretical guarantees, works with any off-the-shelf policy model, and can be added as a lightweight plug‑and‑play module to existing offline RL frameworks. Experiments on OGBench and D4RL show significant gains in performance, robustness, and out‑of‑distribution generalization, especially in low‑data settings.

By Peng Cheng, Yunxian Hou, Zhi Zhou, Qian Zhang, Chang Huang, Xianyuan Zhan