BARC-Bench is a new benchmark comprising 110 tasks across nine simulated workspace environments designed to expose the safety risks of underspecified user instructions for agents. The benchmark evaluates whether agents respect safety boundaries defined by workspace records, implicit environmental context, and unstated social norms, such as avoiding premature disclosure of an employee’s resignation. Results show that even the best model configurations fail to consistently act safely, with safe completion rates ranging from 9.1% to 72.7% and a 27.3% safety violation rate in the strongest setup.
arXiv:2602.03100v2 Announce Type: replace Abstract: Large Language Models (LLMs) are increasingly deployed as agents that operate in real-world environments, introducing safety risks beyond linguisti...
arXiv:2606.00341v2 Announce Type: replace-cross Abstract: As AI agents are increasingly deployed in real personal and corporate settings (email accounts, development workflows, company databases, etc...
arXiv:2609.06783v1 Announce Type: cross Abstract: LLM agents operate in workflows where unsafe actions can have real consequences. Existing safety evaluations often reduce behavior to a single score,...
arXiv:2610.35188v1 Announce Type: cross Abstract: Large language models increasingly act through code, modifying repositories, executing programs, and interacting with services in agent form. Their a...
The paper investigates whether tool‑calling large language model agents maintain consistent safety throughout a conversation. It finds that agents are most vulnerable at the very start of a session, with safety improving significantly after completing a few regular agentic tasks—a phenomenon termed the cold‑start safety gap. The authors introduce the Safety Over Depth for Agents (SODA) benchmark to systematically study this effect, evaluate multiple models, and demonstrate that warming up agents with regular tasks before deployment enhances safety while preserving utility.
arXiv:2610.17685v1 Announce Type: cross Abstract: Enterprise agents face safety risks that cannot be determined from individual actions alone. Existing safety benchmarks largely evaluate action-local...
arXiv:2606. 02965v1 Announce Type: new Abstract: Benchmarks for autonomous agents measure whether agents complete tasks, yet this framing is systematically blind to whether an agent should have proceeded at all.
arXiv:2606. 00341v1 Announce Type: cross Abstract: As AI agents are increasingly deployed in real personal and corporate settings (email accounts, development workflows, company databases, etc.
arXiv:2608. 08601v1 Announce Type: new Abstract: To anticipate socio-technical risks from AI agents, organizations need taxonomies to classify them.
arXiv:2609.39050v1 Announce Type: cross Abstract: As multi-agent systems enter high-stakes domains, the possibility that agents may circumvent safety boundaries is a growing concern. Prior work has e...
Guard models are used to prevent LLM agents from performing forbidden actions, but this alone does not guarantee safety. The paper shows that many agent trajectories contain unfulfilled safety‑critical obligations—actions that should have been performed—highlighting a new source of risk. To address this, the authors introduce ObligationBench, a benchmark of 240 expert‑validated trajectories, and develop ObligationGuard, a model that improves obligation detection recall and exact‑match rates over existing models.
LatentRiskBench is a new benchmark designed to evaluate state-dependent safety risks in enterprise agents, where the safety of an action depends on the workflow state rather than the action alone. It pairs risky trajectories with benign ones that share the same terminal action but differ in latent policy-relevant states, forcing models to reconstruct context instead of relying on action cues. The benchmark introduces residual risk metrics and demonstrates that state-level evaluation detects 96% of violations compared to only 9% for action-level methods, highlighting the inadequacy of existing guardrails.