arXiv:2606. 05888v1 Announce Type: new Abstract: Retry-based objectives such as pass@K and max@K optimize the best return obtained from multiple sampled trajectories, and recent work has shown that they can promote exploration without explicit exploration bonuses.
By Soichiro Nishimori, Paavo Parmas
arXiv:2606. 25012v1 Announce Type: new Abstract: Many reinforcement learning (RL) problems in the infinite-horizon average-reward setting require optimizing multiple conflicting objectives while satisfying multiple safety constraints.
By Ankur Naskar, Swetha Ganesh, Vaneet Aggarwal
arXiv:2608. 03562v1 Announce Type: new Abstract: Reinforcement learning (RL) with general utility extends classic RL by optimizing an arbitrary utility functional of the policy-induced occupancy measure, thereby enabling a broader range of applications.
By Zixuan Liu, Fangzheng Wu, Brian Summa, Zizhan Zheng
The paper introduces Zeroth-Order Log-Barrier Policy Gradient (ZO‑LBPG), a reward‑model‑free method for safe reinforcement learning from human feedback (Safe‑RLHF). ZO‑LBPG optimizes a log‑barrier objective using zeroth‑order estimates derived directly from human preferences, and incorporates conservative constraint margins, coupled rollouts, and an adaptive step size to keep every policy executed during training strictly feasible. The authors prove high‑probability feasibility of all iterates and exploration policies, establish global convergence to an approximately optimal constrained policy, and provide sample complexity bounds for trajectories and human queries, with experimental validation on a synthetic CMDP and a language‑model alignment task.
The paper introduces Exchange Policy Optimization (EPO), a framework for semi‑infinite safe reinforcement learning that handles infinitely many constraints by iteratively solving finite subproblems. EPO expands or deletes constraints based on tolerance violations and Lagrange multipliers, maintaining computational tractability while converging to an optimal policy with bounded safety violations. The authors prove finite convergence, provide iteration bounds, and quantify the suboptimality gap under mild assumptions.
By Jiaming Zhang, Yujie Yang, Haoning Wang, Liping Zhang, Shengbo Eben Li
The paper introduces Zeroth-Order Log-Barrier Policy Gradient (ZO‑LBPG), a reward‑model‑free method for safe reinforcement learning from human feedback (Safe‑RLHF). ZO‑LBPG optimizes a log‑barrier objective using zeroth‑order estimates from human preferences, ensuring that every policy executed during training remains strictly feasible. The authors prove high‑probability feasibility and global convergence, provide sample complexity bounds, and validate the approach on a synthetic CMDP and a language‑model alignment task.
By Weiqin Chen, Xinmeng Huang, Dongsheng Ding, Santiago Paternain