arXiv:2609.22476v1 Announce Type: cross Abstract: Transmission system operators face rising complexity from renewable integration, reduced inertia, and tighter security margins. Large language models...
SentryBench is a cost‑sensitive benchmark designed to evaluate large language model agents that use tools in public‑safety contexts. It measures agents on a frontier that jointly considers attack success, over‑refusal (missing real emergencies), and over‑reaction (unwarranted actions) under adversarial tool use, using scenarios grounded in real emergency‑call transcripts and disaster messages. The study shows that naive caution leads to high harm, while a targeted trust‑gate wrapper reduces harm across models and defenses.
arXiv:2607. 26791v1 Announce Type: cross Abstract: Large Language Model (LLM) agents are increasingly adopted in real-world security operations with access to host artifacts and command-line interfaces (CLIs), making it critical to thoroughly assess their security capabilities.
arXiv:2606. 12797v1 Announce Type: new Abstract: Agentic large language model systems that autonomously invoke tools, maintain persistent memory, and execute multi-step plans are increasingly deployed in public-facing domains, including government services, healthcare triage, and financial advising.
arXiv:2609.06972v1 Announce Type: cross Abstract: LLM agents complete tasks by issuing sequences of tool calls, and every observation they read is a channel through which an indirect prompt injection...
arXiv:2606.00497v2 Announce Type: replace-cross Abstract: Deceptive web content, widely instantiated across the internet and commonly known as \textit{social-engineering attacks}, manipulates autonom...
arXiv:2605.09504v3 Announce Type: replace-cross Abstract: We show that the instruments used to measure AI offensive capability fail in two ways: (i) they overstate harm, and (ii) they credit the mode...
arXiv:2607. 20982v1 Announce Type: new Abstract: As large language model agents increasingly operate autonomously with access to tools and external environments, ensuring their safe and reliable behavior becomes critical.
arXiv:2607. 14570v1 Announce Type: new Abstract: AI software development agents are increasingly capable of modifying infrastructure and security critical systems, creating risks where an agent completes its assigned task while covertly weakening safeguards through actions such as broadening permissions, degrading logging, or introducing persistence mechanisms.
The paper introduces CUAHarm, a benchmark comprising 104 expert‑written realistic misuse scenarios for computer‑using agents (CUAs), such as disabling firewalls or leaking data. Using a sandbox with verifiable rewards, the authors evaluate frontier language models—including GPT‑5, Claude 4 Sonnet, Gemini 2.5 Pro, Llama‑3.3‑70B, and Mistral Large 2—and find that even without jailbreak prompts, these models can successfully execute many malicious tasks at high rates (e.g., 90% for Gemini 2.5 Pro). The study also shows that newer models, while safer in traditional safety benchmarks, exhibit higher misuse risks as CUAs, and that monitoring CUAs’ actions remains challenging, with current methods achieving only about 77% accuracy.
arXiv:2606. 00925v1 Announce Type: cross Abstract: Open agent platforms allow community contributors to publish reusable skills that agents can invoke at runtime.
MAS-Shield is a defense framework for Large Language Model–based Multi-Agent Systems that uses a coarse‑to‑fine filtering pipeline. It first selects critical agents, then applies lightweight auditing to most cases, and finally escalates only suspicious signals to a heavyweight committee. Experiments show a 92.5% recovery rate against adversarial attacks and a latency reduction of over 70% compared to existing methods.