The paper introduces WSP-CVaR-RLHF, a risk‑sensitive reinforcement learning algorithm that learns from human feedback while handling corrupted or adversarial preference labels. It uses uncertainty‑weighted reward estimation and optimistic CVaR planning to achieve a regret bound that separates clean statistical cost from corruption penalties. Experiments with four adversarial attacks show that the method consistently reduces cumulative regret compared to an unweighted robust baseline while maintaining confidence‑set coverage.
By Xinyi Ni, Lifeng Lai
In reinforcement learning policy evaluation, classic on-policy methods often suffer from high variance when estimating policy performance. To mitigate this issue, behavior policy search has been propo...
arXiv:2405.14749v3 Announce Type: replace-cross
Abstract: Risk-sensitive reinforcement learning (RL) is crucial for maintaining reliable performance in high-stakes applications. While traditional RL...
By Xian Yu, Minheng Xiao, Lei Ying
The paper introduces an Onsager‑corrected, target‑policy‑weighted criterion that estimates the out‑of‑sample risk of a reward model trained on the full dataset using only its training data, thereby eliminating the need for sample splitting. This approach allows early stopping along a full‑data gradient‑descent trajectory and provides uniform consistency guarantees for neural networks under Gaussian contexts and single‑index rewards. The correction also yields a doubly robust policy‑value estimator that remains consistent even when reward‑model prediction error does not vanish, and experiments demonstrate reduced risk‑estimation error and improved reward‑model selection compared to held‑out baselines.
whyItMatters":"By removing the need for sample splitting, the method enables more efficient use of data and improves the reliability of off‑policy evaluation and reward‑model selection in high‑dimensional settings."
By Saksham Jain, Guillaume Braun, Masaaki Imaizumi
arXiv:2510. 19528v2 Announce Type: replace-cross Abstract: We investigate the fundamental problem of leveraging offline data to accelerate online reinforcement learning - a direction with strong potential but limited theoretical grounding.
By Sebastian Reboul, H\'el\`ene Halconruy
arXiv:2607. 23030v1 Announce Type: new Abstract: Developing efficient function-approximation methods for policy evaluation is a fundamental challenge in risk-aware reinforcement learning.
By Weikai Wang, Erick Delage
BCPPO is a new variant of Proximal Policy Optimization that uses Bachelier-inspired cost‑prediction networks to generate a smooth penalty based on disagreement among critics. The method keeps temporal‑difference learning unchanged, applies a saturation‑aware controller to manage cost penalties, and deploys only the policy network. Across extensive experiments, BCPPO outperforms comparators in achieving higher mean returns while maintaining lower or comparable CVaR in all tested tasks.
By Dongsheng Hou, Yanqiao Chen, Yuhan Rui
arXiv:2608.24146v1 Announce Type: new
Abstract: In reinforcement learning policy evaluation, classic on-policy methods often suffer from high variance when estimating policy performance. To mitigate...
By Claire Chen, Shuze Daniel Liu, Licheng Luo, Rohan Chandra, Nan Jiang, Shangtong Zhang
arXiv:2610.02198v1 Announce Type: cross
Abstract: Several state-of-the-art methods for online reinforcement learning in continuous control improve policies using action gradients of a learned critic....
By Sebastian Sanokowski, Alireza Sarmadi, Majid Khadiv
The paper presents the first provably efficient offline reinforcement learning algorithms for risk-sensitive settings using optimized certainty equivalents (OCE). It introduces two formulations—dynamic-OCE, applying OCE at each step, and static-OCE, applying a single OCE to cumulative rewards—and develops pessimistic value-iteration methods with tight sublinear suboptimality bounds in tabular and linear function approximation settings. The authors also establish minimax lower bounds, characterizing the fundamental limits of offline risk-sensitive RL.
By Hanzhen Wang, Boxiang Lyu, Zhongxiang Dai, Siyang Gao, Dake Zhang, Shuang Qiu
arXiv:2606. 00320v1 Announce Type: new Abstract: We present an online, distribution-free framework for controlling the Conditional Value-at-Risk (CVaR), extending conformal tail risk control to non-stationary and adversarial environments.
By Catherine Chen, Jingyan Shen, Zhun Deng, Lihua Lei
arXiv:2606. 00913v1 Announce Type: cross Abstract: Multi-arm bandit algorithms are increasingly used in online platforms, clinical trials, and social science experiments, but valid statistical inference on their performance remains an open challenge.
By Samya Praharaj, Chih-Yu Chang, Koulik Khamaru, Kelly W. Zhang