arXiv AI August 26, 2026 By Houcheng Jiang, Boxuan Zhang, Qiyong Zhong, Junfeng Fang, Xiang Wang, Xiangnan He

RePolicy: Reinforcement Learning for Safety-Policy Invocation in Agent Safeguards

RePolicy is a reinforcement learning approach designed to invoke safety policies for language model agents by evaluating entire execution trajectories within context-dependent policy libraries. It generates policy-grounded rationales and safety judgments, and is initialized with the PolicyTraj-20K dataset before fine-tuning via GRPO with verifiable rewards and policy-context perturbation. Experiments on six safety benchmarks demonstrate strong safety-detection performance and robust policy invocation across varying contexts.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Hugging Face Trending Papers

Aug 25

RePolicy: Reinforcement Learning for Safety-Policy Invocation in Agent Safeguards

Safeguarding language model agents requires assessing complete execution trajectories under context-dependent safety policies. Existing policy-aware safeguards mainly rely on prompting or supervised f...

llms agents reinforcement-learning fine-tuning benchmarks safety

arXiv AI

Jun 10

SHAPO: Sharpness-Aware Policy Optimization for Safe Exploration

arXiv:2606. 10228v1 Announce Type: cross Abstract: Safe exploration is a prerequisite for deploying reinforcement learning (RL) agents in safety-critical domains.

By Kaustubh Mani, Yann Pequignot, Vincent Mai, Liam Paull

agents reinforcement-learning safety

arXiv AI

Jun 2

SafeMCP: Proactive Power Regulation for LLM Agent Defense via Environment-Grounded Look-Ahead Reasoning

arXiv:2606. 01991v1 Announce Type: new Abstract: As Large Language Model (LLM) agents increasingly leverage the Model Context Protocol (MCP) to operate in complex environments, the expansion of their action spaces offers agents unsafe capabilities and underscores the risk of power-seeking.

By Lichao Wang, Zhaoxing Ren, Tianzhuo Yang, Jiaming Ji, Chi Harold Liu, Yaodong Yang, Juntao Dai

llms agents reinforcement-learning safety

arXiv AI

Jun 16

Reward Hacking in Language Model Agents: Revisiting AI Safety Gridworlds

arXiv:2606. 15385v1 Announce Type: new Abstract: Reward hacking, where AI systems exploit misspecified objectives to achieve high reward without satisfying intended goals, remains a central challenge in AI safety.

By \"Omer Veysel \c{C}a\u{g}atan, Xuandong Zhao

llms agents reinforcement-learning safety

arXiv AI

3d ago

Evaluation Metrics for Safe Reinforcement Learning

The paper introduces new evaluation metrics for safe reinforcement learning that go beyond average safety guarantees by examining how often and how severely safety bounds are violated, consistency across tasks and bounds, and the relationship between training-time and final policy behavior. It also proposes a safety tier system for categorizing algorithms and presents empirical safety evaluations on multiple navigation tasks. The authors recommend reporting aggregate metrics, distributional data, and task‑specific results together, and provide an open‑source suite, SafeRLEval, to facilitate reliable safety assessment.

By Lindsay Spoor, Aske Plaat, Thomas Moerland

agents reinforcement-learning benchmarks safety

arXiv AI

Jun 6

From Risk Classification to Action Plan Remediation: A Guardrail Feedback Driven Framework for LLM Agents

arXiv:2606. 05805v1 Announce Type: new Abstract: LLM-based guardrails typically safeguard agents by evaluating proposed actions or inputs before execution, producing safety signals such as binary allow/deny decisions, risk categories, and/or explanatory rationales about potential policy violations.

By Yuhao Sun, Jiacheng Zhang, Shaanan Cohney, Zhexin Zhang, Feng Liu, Xingliang Yuan

llms agents safety

One email a morning, machine-written

Related stories

RePolicy: Reinforcement Learning for Safety-Policy Invocation in Agent Safeguards

SHAPO: Sharpness-Aware Policy Optimization for Safe Exploration

SafeMCP: Proactive Power Regulation for LLM Agent Defense via Environment-Grounded Look-Ahead Reasoning

Reward Hacking in Language Model Agents: Revisiting AI Safety Gridworlds

Evaluation Metrics for Safe Reinforcement Learning

From Risk Classification to Action Plan Remediation: A Guardrail Feedback Driven Framework for LLM Agents