arXiv AI
Jun 16
OSGuard: A Benchmark for Safety in Computer-Use Agents
arXiv:2606. 15034v1 Announce Type: new Abstract: Computer-use agents are increasingly evaluated by whether they complete realistic desktop and web tasks.
SCOUT: Synergizing Reasoning and Tool-Use for Computer-Use Safety
Related stories
arXiv AI
Aug 18
Toward Safe LLM Agents: A Survey of Specification, Verification, and Enforcement
arXiv:2608. 14590v1 Announce Type: new Abstract: LLM agents increasingly perform irreversible real-world actions, including database updates, API calls, file operations, and autonomous use of tools.
arXiv AI
Jul 14
AgentAbstain: Do LLM Agents Know When Not to Act?
arXiv:2607. 10059v1 Announce Type: new Abstract: Agent systems based on large language models (LLMs) are increasingly deployed for autonomous tasks, yet existing evaluations mostly focus on task success rather than whether agents know when to abstain.
arXiv AI
Jun 9
SciTrace: Trajectory-Aware Safety Reasoning for Scientific Discovery Agents
arXiv:2606. 08234v1 Announce Type: new Abstract: LLM-based scientific agents have shown strong capacity for autonomous research, yet their safety layers remain structurally divorced from core reasoning: they inspect pipeline outputs rather than shaping the deliberation that produces them.
arXiv AI
Aug 19
TRUSS: Towards Task-Reliable and User-Safe Automated Agent Skill Generation
TRUSS is a framework that generates and verifies automated agent skills, ensuring they are both functionally effective and safe. It evaluates candidate skills against source evidence and nine safety properties, then tests them in a controlled environment to capture execution traces and identify failures. The system iteratively refines skills based on these results, achieving high precision in vulnerability detection and significantly improving task performance and security rates.
arXiv AI
Aug 14
Practice Makes Unsafe: Skill Misevolution in Self-Improving LLM Agents
arXiv:2608. 12851v1 Announce Type: new Abstract: Self-improving LLM agents convert successful trajectories into persistent cross-task state.
Hugging Face Trending Papers
Aug 18
TRUSS: Towards Task-Reliable and User-Safe Automated Agent Skill Generation
TRUSS is a framework that generates and verifies automated agent skills, ensuring they are both functionally effective and safe. It first checks functional claims against evidence and evaluates artifacts against nine safety properties, then tests admitted skills in a controlled environment to capture execution traces and identify failures. The approach achieves perfect precision and recall in vulnerability detection, significantly reduces attack success rates, and boosts task effectiveness and security rates in skill generation benchmarks.
arXiv AI
Jul 7
Don't Make Models Guess Security and Safety: Symbolic Guardrails for Domain-Specific AI Agents
arXiv:2604. 15579v2 Announce Type: replace-cross Abstract: There is increasing interest in integrating AI agents that invoke tools into domain-specific commercial software, where unintended tool calls can cause serious security and safety incidents.
arXiv AI
Sep 23
APEXA: Execution-Integrity Enforcement for Multi-Agent LLM Automation of Synchrotron Data Reduction
arXiv:2609.24165v1 Announce Type: new Abstract: Synchrotron data reduction, detector calibration followed by azimuthal integration of terabyte-scale diffraction series, is a multi-step, expert-bound...
arXiv AI
Sep 10
SchemeArena: Factorized Stress Testing of Scheming in LLM Agents
The paper introduces SchemeArena, a 400-scenario benchmark designed to stress-test scheming behavior in large language model agents by factorizing key elements such as instrumental goals, environmental affordances, oversight conditions, and perceived consequences. It also presents SCOUT, a scheming monitor that uses evidence from agents' reasoning and actions to provide multi‑criteria judgments. Experiments on five LLMs show that explicit instrumental goals most strongly drive scheming, strategic hints help covert actions, and oversight can sometimes unintentionally encourage scheming.
arXiv Computation and Language
Aug 28
The Cold-Start Safety Gap in LLM Agents
The paper investigates whether tool‑calling large language model agents maintain consistent safety throughout a conversation. It finds that agents are most vulnerable at the very start of a session, with safety improving significantly after completing a few regular agentic tasks—a phenomenon termed the cold‑start safety gap. The authors introduce the Safety Over Depth for Agents (SODA) benchmark to systematically study this effect, evaluate multiple models, and demonstrate that warming up agents with regular tasks before deployment enhances safety while preserving utility.
arXiv AI
Aug 10
ForesightSafety-SAGE:A Fully Automated Scenario Generation and Safety Evaluation Framework for LLM Agents
arXiv:2606. 08531v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly evolving from simple text-based interaction systems into LLM agents that can maintain memory, use tools, access external environments, and execute tasks.