Safeguarding language model agents requires assessing complete execution trajectories under context-dependent safety policies. Existing policy-aware safeguards mainly rely on prompting or supervised f...
RePolicy is a safeguard that learns to both select the appropriate safety policy from a library and assess agent behavior against it. It processes an agent trajectory and a set of candidate policies, chooses the best-matching policy, applies its requirements, and predicts safety. The authors created PolicyTraj-20K, a dataset of over 20,000 annotated trajectories, and trained RePolicy with supervised and reinforcement learning, achieving top performance on six benchmarks and 19 baselines.
By Houcheng Jiang, Boxuan Zhang, Qiyong Zhong, Junfeng Fang, Xiang Wang, Xiangnan He
arXiv:2606. 10228v1 Announce Type: cross Abstract: Safe exploration is a prerequisite for deploying reinforcement learning (RL) agents in safety-critical domains.
By Kaustubh Mani, Yann Pequignot, Vincent Mai, Liam Paull
PROACT-Agent is a framework designed to provide proactive runtime oversight for large language model agents, aiming to prevent irreversible environmental harm by enabling real‑time guardrails. It introduces three key components: Progressive Trajectory Unrolling, Reasoning‑Augmented Causal Rectification, and Culturally‑Aware Data Localization, which together address safety drift and enforce temporal consistency. The accompanying PROACT‑Bench, a bilingual safety benchmark with 155,780 labeled states, demonstrates the framework’s effectiveness, achieving high unsafe‑class F1 scores and significantly reducing attack success rates in AgentDojo.
By Ding Jia, Wei Liu, Xianglong Du, Yingjie Li, Yingqing Yang, Huili Yu, Zhangsong Zhan, Chu Zhou
arXiv:2606. 01991v1 Announce Type: new Abstract: As Large Language Model (LLM) agents increasingly leverage the Model Context Protocol (MCP) to operate in complex environments, the expansion of their action spaces offers agents unsafe capabilities and underscores the risk of power-seeking.
By Lichao Wang, Zhaoxing Ren, Tianzhuo Yang, Jiaming Ji, Chi Harold Liu, Yaodong Yang, Juntao Dai
The paper introduces Safety to Competence (S2C), a two‑stage reinforcement learning framework that first learns a safety filter and then trains a competitive task policy while embedding the filter. By separating safety synthesis from task learning, S2C reduces training complexity and prevents the policy from being exploited by adversarial attacks. Experiments on simulated touchdown games show that S2C achieves higher win rates, better Elo ratings, and lower exploitability than eight safe‑RL baselines, and hardware tests confirm its competence against a human opponent.
By Ruihan Wu, Rui Yang, Donggeon David Oh, Duy Nguyen, Haimin Hu