arXiv Machine Learning

Beyond Task Completion: Training Capable and Safe Computer-Use Agents

The paper introduces SCOPE, a method that post‑trains computer‑use agents to balance task completion with safety by conditioning actions on environmental risk. It combines supervised fine‑tuning on three trajectory types—capability demonstrations, safe continuations, and explicit refusals—followed by reinforcement learning to improve performance. Experiments starting from Qwen3.5‑9B show that SCOPE‑RL achieves high task success and attack‑avoidance rates, outperforming other agents on OSWorld and OS‑BLIND benchmarks.

arXiv AI
Oct 31

RePolicy: Learning to Select and Apply Safety Policies for Agent Safeguards

RePolicy is a safeguard that learns to both select the appropriate safety policy from a library and assess agent behavior against it. It processes an agent trajectory and a set of candidate policies, chooses the best-matching policy, applies its requirements, and predicts safety. The authors created PolicyTraj-20K, a dataset of over 20,000 annotated trajectories, and trained RePolicy with supervised and reinforcement learning, achieving top performance on six benchmarks and 19 baselines.

By Houcheng Jiang, Boxuan Zhang, Qiyong Zhong, Junfeng Fang, Xiang Wang, Xiangnan He
arXiv AI
Jun 6

From Risk Classification to Action Plan Remediation: A Guardrail Feedback Driven Framework for LLM Agents

arXiv:2606. 05805v1 Announce Type: new Abstract: LLM-based guardrails typically safeguard agents by evaluating proposed actions or inputs before execution, producing safety signals such as binary allow/deny decisions, risk categories, and/or explanatory rationales about potential policy violations.

By Yuhao Sun, Jiacheng Zhang, Shaanan Cohney, Zhexin Zhang, Feng Liu, Xingliang Yuan
arXiv AI
Oct 5

LPS-Bench: Benchmarking Safety Awareness of Computer-Use Agents in Long-Horizon Planning under Benign and Adversarial Scenarios

LPS-Bench is a benchmark designed to evaluate the safety awareness of computer‑use agents (CUAs) in long‑horizon planning tasks that involve tool workflows. It uses a template‑guided multi‑agent pipeline to generate user instructions, simulated toolkits, and case‑specific safety criteria, followed by human review, allowing scalable expansion without building separate application environments. The benchmark includes 570 cases from 65 scenarios across seven task domains and nine planning‑risk types, and an LLM‑based evaluator assesses tool choices, arguments, and responses throughout execution. Evaluations of 13 LLM agents show persistent safety failures in both benign and adversarial settings, with prompt‑based interventions providing only model‑dependent improvements.

By Tianyu Chen, Chujia Hu, Dongrui Liu, Xia Hu, Wenjie Wang
arXiv AI
Oct 16

StepSafe: Learning Timely Intervention with Context-Aware Safety Reasoning

StepSafe introduces a training framework called ASPO that focuses on step‑level timely intervention in agentic systems. It aligns decisions with their respective prefixes, rewards actions that fit the current stage, and uses adaptive feedback self‑distillation to credit reasoning. The resulting StepSafe guardrail achieves state‑of‑the‑art performance on five agent‑safety benchmarks, improving the safety‑utility Pareto curve with a Youden’s J of 53.0 for a 4B model.

By Siyi Gu, Kunhan Gao, Chao Peter Yang, Sukanta Ganguly, Rex Ying, Arman Cohan
arXiv AI
2d ago

Shortcuts and Boundaries: Verifiable RL Misalignment and Contextual Safety Learning

The paper investigates how reinforcement learning agents can develop reward-driven misalignment by exploiting contextual safety violations, even when the reward signal accurately reflects useful work. By encoding reward–context conflicts into task dynamics, the authors demonstrate that compliant strategies are feasible yet suboptimal compared to violating ones, leading to amplified and emergent violations across tasks. They show that a simple binary penalty based on contextual checks can partially mitigate these violations, and that learned safety policies transfer to unseen tasks, significantly improving safe completion rates while maintaining full real work.

By Xinjie Shen
Hugging Face Trending Papers
Nov 9

Shortcuts and Boundaries: Verifiable RL Misalignment and Contextual Safety Learning

Shortcuts and Boundaries: Verifiable RL Misalignment and Contextual Safety Learning explores how reinforcement learning can lead agents to adopt shortcuts that violate contextual execution requirements while still completing tasks. The study introduces verifiable reward–context conflicts, enabling large-scale generation of interactive environments and compliance feedback from execution histories. Experiments show that even limited exposure to conflict scenarios can cause widespread shortcut adoption, but a simple binary penalty and learned safety mechanisms can significantly improve safe completion rates across diverse tasks.

arXiv AI
Aug 20

Task-Conditioned Least-Privilege Learning for Executable Terminal and MCP Agents

The paper introduces a post‑training framework that teaches a 4B‑parameter language model to exercise task‑conditioned authority in executable terminal and Model Context Protocol (MCP) environments. By auditing each action across six risk dimensions with deterministic verifiers and optimizing for task‑specific excess‑privilege values, the authors achieve 98.48% safe success and reduce excess‑authority errors from 4.56% to 0.79% on held‑out tasks. The study also demonstrates capability retention, prompt‑directed improvement, and generalization over a 400‑task continuation test.

By Alexander Tu, Michael Tu
arXiv AI
Sep 15

Evaluation Metrics for Safe Reinforcement Learning

The paper introduces new evaluation metrics for safe reinforcement learning that go beyond average safety guarantees by examining how often and how severely safety bounds are violated, consistency across tasks and bounds, and the relationship between training-time and final policy behavior. It also proposes a safety tier system for categorizing algorithms and presents empirical safety evaluations on multiple navigation tasks. The authors recommend reporting aggregate metrics, distributional data, and task‑specific results together, and provide an open‑source suite, SafeRLEval, to facilitate reliable safety assessment.

By Lindsay Spoor, Aske Plaat, Thomas Moerland