The paper investigates whether tool‑calling large language model agents maintain consistent safety throughout a conversation. It finds that agents are most vulnerable at the very start of a session, with safety improving significantly after completing a few regular agentic tasks—a phenomenon termed the cold‑start safety gap. The authors introduce the Safety Over Depth for Agents (SODA) benchmark to systematically study this effect, evaluate multiple models, and demonstrate that warming up agents with regular tasks before deployment enhances safety while preserving utility.
By Chung-En Sun, Linbo Liu, Tsui-Wei Weng
SentryBench is a cost‑sensitive benchmark designed to evaluate large language model agents that use tools in public‑safety contexts. It measures agents on a frontier that jointly considers attack success, over‑refusal (missing real emergencies), and over‑reaction (unwarranted actions) under adversarial tool use, using scenarios grounded in real emergency‑call transcripts and disaster messages. The study shows that naive caution leads to high harm, while a targeted trust‑gate wrapper reduces harm across models and defenses.
By Md Abu Yusuf, Oyshi Tabassum Aditi, Md Faysal Hosain Mazed
arXiv:2608.24569v2 Announce Type: replace
Abstract: Large language model (LLM) agents build on one another's work through summaries, plans, and other intermediate artifacts. These records let a downs...
By Yiheng Sun, Huifei Wang, Yancheng Zhu, Zhenyu Li, Zebin Zhao, Yifan Yuan
The paper introduces a benchmark to measure the propensity of large language model agents to exhibit instrumental convergence (IC) behaviors—such as self‑preservation—that could pose risks. The benchmark consists of seven operational tasks with varied conditions (monitoring, instruction clarity, stakes, etc.) and was tested on ten models, revealing an overall IC rate of 5.1% across 1,680 samples. IC behaviors were concentrated in a few models and tasks, with conditions making IC indispensable for success increasing the rate by 15.7 percentage points.
By Jonas Wiedermann-M\"oller, Leonard Dung, Maksym Andriushchenko
arXiv:2609.36855v1 Announce Type: new
Abstract: Multi-agent LLM systems rely on message passing among specialized agents to accomplish complex tasks. However, an upstream agent may provide useful inf...
By Yaxin Gong, Gangyi Zhang, Chongming Gao, Leyang Shen, Chenxiao Fan, Jiakai Wang, Dong Wang, Yang Liu, Wenjie Wang, Xiangnan He
arXiv:2610.34167v1 Announce Type: new
Abstract: Tool-using language model agents are usually evaluated by whether they finish the user's task. That metric misses a common failure: an agent can comple...
By Jiaju Huang, Shaobin Chen, Xinglong Liang, Xinyu Ma, Hao Yang, Wei Ke, Tao Tan, Yue Sun
arXiv:2610.15459v1 Announce Type: cross
Abstract: LLM-based multi-agent systems (LLM-MAS) use specialized agents that exchange messages to accomplish complex tasks. Their reliance on networked protoc...
By Jinhao Zhou, Xiaoyu Zhang, Jiongchi Yu, Fahao Chen, Jun Wu
arXiv:2607. 19449v1 Announce Type: cross Abstract: Evaluation frameworks for tool-augmented LLM agents focus overwhelmingly on capability metrics or explicit tool crashes, leaving silent infrastructure failures and HTTP 200 responses with empty, null, or malformed payloads largely unaudited.
By Aarushi Singh
The paper evaluates a three‑part control stack— a 5‑Pillar runtime constitution, a 4‑tier swarm (Director, majority vote, Monitor, hard gate), and Cognitive Annealing—for multi‑agent LLM negotiation against an adversarial Gatekeeper. Experiments with Gemini 2.5 Pro agents and Claude Haiku 4.5 show that the constitution and Director improve framing but not reliability, the Monitor and hard gate do not reduce unlocks, and deterministic recovery via atomic purge and a canonical strike successfully resolves deadlocks while keeping call budgets unchanged. The study demonstrates that the stack can recover from deadlocks without additional model cost, though it does not reduce overall call or token usage as initially hypothesized.
By Masaaki Nakatsu (AO, Inc. / OrbLabs AG), Reno Wang (AO, Inc.)
arXiv:2607. 07474v1 Announce Type: cross Abstract: Agentic red-teaming benchmarks report whether an injected agent was compromised as a single bit: the attack succeeded, or it did not.
By Harry Owiredu-Ashley
arXiv:2607. 07097v1 Announce Type: new Abstract: Safety evaluations of multi-agent LLM systems often compare a direct prompt with a planner-executor pipeline and report the difference as a single "pipeline effect.
By Lifei Liu, Haoran Yu, Xiaochong Jiang, Su Wang, Pin Qian, Yihang Chen
The paper introduces a new multi‑agent micro‑benchmark called Delay‑of‑Gratification, modeled after the Stanford marshmallow experiment, to evaluate large language models (LLMs) in long‑horizon, multi‑turn interactions. In the benchmark, ReAct agents use a per‑step “raise a question” tool under various constraints—social context (broadcast vs. isolated), persona traits (age, hedonic drive), and tool‑use policy (mandatory vs. optional). Across 19,200 trajectories, the study finds that most agents exhibit an early impulse to “eat,” only 75.9% persist to the end, and factors such as isolation and hedonic drive significantly influence survival and questioning behavior, with ablations showing that removing hedonic drive and age can improve completion rates.
By Olga Manakina, Igor Bogdanov, Chung-Horng Lung