← Back to all news
arXiv AI By Pekka Malo, Lauri Viitasaari, Antti Suominen, Eeva Vilkkumaa, Olli Tahvonen

The Price of Failure Risk: Clipped Barrier Policy Gradients with Convex Regularization

Read the original on arXiv AI →

The paper introduces a clipped barrier policy gradient method that incorporates a convex regularizer to manage failure risk in budget-constrained operations. By adding a barrier that grows as the budget depletes and clipping rewards below a threshold, the approach ensures that the learned policy maximizes utility while keeping risk within a bounded margin. The authors prove exponential convergence of the training dynamics under convex regularization and demonstrate the method on several benchmark tasks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Sep 1

BCPPO: Bachelier-Inspired Constrained Proximal Policy Optimization for Tail-Risk-Aware Safe Reinforcement Learning

BCPPO is a new variant of Proximal Policy Optimization that uses Bachelier-inspired cost‑prediction networks to generate a smooth penalty based on disagreement among critics. The method keeps temporal‑difference learning unchanged, applies a saturation‑aware controller to manage cost penalties, and deploys only the policy network. Across extensive experiments, BCPPO outperforms comparators in achieving higher mean returns while maintaining lower or comparable CVaR in all tested tasks.

By Dongsheng Hou, Yanqiao Chen, Yuhan Rui
reinforcement-learningsafety
More like this →
arXiv Machine Learning
Sep 17

A Convergence Framework for Deep $V$-Learning: Error Propagation and Sharp Action-Gap Bounds

The paper presents a convergence framework for deep $V$‑learning over a finite horizon $H$, deriving explicit bounds on policy loss by decomposing the Bellman update error into six residuals. It shows how $L^s$ concentrability controls expected $L^1$ loss, quantifies the impact of shared sampling across horizon levels, and provides optimal and near‑optimal sample allocations for statistical error rates. The work also establishes sharp action‑gap bounds under a margin condition, transfers optimal‑gap results to frozen‑iterate gaps, and offers consistency guarantees for generative‑reset approximate‑ERM procedures with exact action scores.

By Yury Kolomeytsev
More like this →
arXiv Machine Learning
Oct 5

Planning to Learn

The paper introduces a new loss function called the horizon loss for training classifiers. It argues that the exact policy gradient used in reinforcement learning is myopic, whereas cross‑entropy is patient, and the horizon loss interpolates between the two by truncating the total error at the remaining learning. Experiments on MNIST and ImageNet with ResNet and ViT models show that horizon loss consistently improves top‑1 accuracy over cross‑entropy, especially when label noise is present.

By Ian Osband
llmsreinforcement-learning
More like this →