The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
arXiv:2611.07561v1 Announce Type: new Abstract: Safe reinforcement learning from human feedback (Safe-RLHF) seeks to improve helpfulness while satisfying harmlessness constraints. Existing primal-dua...
The paper introduces two algorithms for safe multi‑objective reinforcement learning that guarantee zero constraint violation during learning. The first algorithm learns a safe Pareto‑optimal policy for a given preference by combining optimistic reward estimates with pessimistic cost bounds, achieving an ×O(1/√T) suboptimality gap. The second, preference‑free algorithm safely explores first and then computes a safe policy for any preference, retaining the same safety guarantee and achieving an ×O(1/√T+1/√K) suboptimality gap.
arXiv:2608. 10204v1 Announce Type: new Abstract: Safe reinforcement learning maximizes reward subject to safety constraints.
The paper introduces Policy Gradient Penalty (PGP), a single‑loop policy‑space method that enforces convex occupancy‑measure constraints via quadratic‑penalty regularization. PGP constructs pseudo‑rewards to estimate gradients of the penalized objective and uses the classical Policy Gradient Theorem, establishing smoothness and global last‑iterate convergence guarantees for an ε‑optimal constrained entropy value with ε‑bounded constraint violation. The authors validate PGP with ablations on a grid‑world benchmark and demonstrate scalability on two challenging continuous‑control tasks.
The paper introduces Exchange Policy Optimization (EPO), a framework for semi‑infinite safe reinforcement learning that handles infinitely many constraints by iteratively solving finite subproblems. EPO expands or deletes constraints based on tolerance violations and Lagrange multipliers, maintaining computational tractability while converging to an optimal policy with bounded safety violations. The authors prove finite convergence, provide iteration bounds, and quantify the suboptimality gap under mild assumptions.
arXiv:2607. 15457v1 Announce Type: new Abstract: We study robust peak-cost constrained reinforcement learning (RP-CRL), where the objective is to maximize expected reward while controlling the maximum cost encountered along a trajectory.