arXiv Machine Learning
Provable Zeroth-Order Safe Exploration for Reinforcement Learning from Human Feedback
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
Hugging Face Trending Papers
1d ago
Provable Zeroth-Order Safe Exploration for Reinforcement Learning from Human Feedback
Safe reinforcement learning from human feedback (Safe-RLHF) seeks to improve helpfulness while satisfying harmlessness constraints. Existing primal-dual approaches can converge to feasible policies bu...
arXiv Machine Learning
Aug 12
Boundary-Seeking Policy Gradient for Safe Reinforcement Learning
arXiv:2608. 10204v1 Announce Type: new Abstract: Safe reinforcement learning maximizes reward subject to safety constraints.
arXiv AI
Oct 31
Safe Multi-Objective Reinforcement Learning: Zero Constraint Violation and Primal-Averaging Convergence
The paper introduces two algorithms for safe multi‑objective reinforcement learning that guarantee zero constraint violation during learning. The first algorithm learns a safe Pareto‑optimal policy for a given preference by combining optimistic reward estimates with pessimistic cost bounds, achieving an ×O(1/√T) suboptimality gap. The second, preference‑free algorithm safely explores first and then computes a safe policy for any preference, retaining the same safety guarantee and achieving an ×O(1/√T+1/√K) suboptimality gap.
arXiv Machine Learning
Sep 30
Global Optimality for Constrained Exploration via Penalty Regularization
The paper introduces Policy Gradient Penalty (PGP), a single‑loop policy‑space method that enforces convex occupancy‑measure constraints via quadratic‑penalty regularization. PGP constructs pseudo‑rewards to estimate gradients of the penalized objective and uses the classical Policy Gradient Theorem, establishing smoothness and global last‑iterate convergence guarantees for an ε‑optimal constrained entropy value with ε‑bounded constraint violation. The authors validate PGP with ablations on a grid‑world benchmark and demonstrate scalability on two challenging continuous‑control tasks.
arXiv Machine Learning
Sep 3
Exchange Policy Optimization Algorithm for Semi-Infinite Safe Reinforcement Learning
The paper introduces Exchange Policy Optimization (EPO), a framework for semi‑infinite safe reinforcement learning that handles infinitely many constraints by iteratively solving finite subproblems. EPO expands or deletes constraints based on tolerance violations and Lagrange multipliers, maintaining computational tractability while converging to an optimal policy with bounded safety violations. The authors prove finite convergence, provide iteration bounds, and quantify the suboptimality gap under mild assumptions.
arXiv Machine Learning
Jul 9
UP: Unbounded Positive Asymmetric Optimization for Breaking the Exploration-Stability Dilemma
arXiv:2607. 06987v1 Announce Type: new Abstract: Reinforcement learning (RL) has become the standard paradigm for enhancing the complex reasoning capabilities of large language models (LLMs).