arXiv AI

Generalization in offline RL: The structure is more important than the amount of pessimism

arXiv:2607. 02288v1 Announce Type: cross Abstract: While pessimism counteracts overestimation bias in offline reinforcement learning (RL), being overly conservative has been associated with hindering certain forms of generalization.

arXiv Machine Learning
Sep 18

Improving Generalization and Robustness in Offline Reinforcement Learning via Boundary-Aware Data Augmentation

The paper introduces BADA, a Boundary-Aware Data Augmentation technique for offline reinforcement learning. By interpolating neighboring states to create synthetic data that respects the original distribution, BADA improves in-distribution generalization and robustness. Experiments on limited offline datasets show that BADA achieves state-of-the-art performance across diverse benchmarks.

By Gong Gao, Weidong Zhao, Xianhui Liu
arXiv Machine Learning
Aug 4

Diffusion Policy with Behavioral Advantage Correction for Offline Reinforcement Learning

arXiv:2608. 02332v1 Announce Type: new Abstract: In offline reinforcement learning (RL), the distribution shift between behavioral data and the learned policy can lead to erroneous \emph{Q}-value estimation, thereby misguiding the direction of policy optimization.

By Botao Dong, Longyang Huang, Ning Pang, Hongtian Chen