The paper investigates whether tool‑calling large language model agents maintain consistent safety throughout a conversation. It finds that agents are most vulnerable at the very start of a session, with safety improving significantly after completing a few regular agentic tasks—a phenomenon termed the cold‑start safety gap. The authors introduce the Safety Over Depth for Agents (SODA) benchmark to systematically study this effect, evaluate multiple models, and demonstrate that warming up agents with regular tasks before deployment enhances safety while preserving utility.
By Chung-En Sun, Linbo Liu, Tsui-Wei Weng
SIGMA is a self‑improving alignment framework that uses a model’s own reasoning to generate diverse alignment dilemma scenarios from a simple "Model Spec" and then fine‑tunes the model with self‑judged rewards. By performing spec‑guided task synthesis and rubric‑based reinforcement learning, SIGMA improves safety alignment in multi‑turn agentic settings, reducing harmfulness and misalignment scores while preserving general capability. The approach demonstrates that a model can autonomously strengthen its own safety reasoning without external supervision.
By Jingyu Zhang, Shruti Palaskar, Daniel Khashabi, Benjamin Van Durme, Leon A. Gatys, Joseph Yitan Cheng
arXiv:2609.27900v1 Announce Type: new
Abstract: Large language models (LLMs) are increasingly deployed in multi-agent systems where a principal agent decomposes tasks and delegates them to subordinat...
By Zonghao Ying, Jiaqi Yan, Huize Luo, Quanchen Zou, Aishan Liu, Xianglong Liu
arXiv:2607. 05775v1 Announce Type: new Abstract: Large language model (LLM) agents are increasingly evaluated on their ability to use tools, plan multi-step tasks, coordinate with other agents, and operate over extended horizons.
By Wael Albayaydh, Rui Zhao, Ivan Flechais
arXiv:2606. 02282v1 Announce Type: new Abstract: Orchestrating Large Language Models into Multi-Agent Systems (LLM-MAS) has unlocked remarkable reasoning capabilities, yet emergent failures and hallucinations that resist characterisation block their deployment in safety-critical domains -- a gap made legally untenable by emerging AI regulation.
By I\~naki Dellibarda Varela, R. Sendra-Arranz, Pablo Romero-Sorozabal, J. M. Valverde-Garc\'ia, Annemarie F. Laudanski, \'Alvaro Guti\'errez, Eduardo Rocon, Manuel Cebrian
Large language model (LLM) agents are increasingly evaluated on their ability to use tools, plan multi-step tasks, coordinate with other agents, and operate over extended horizons. Reported benchmark gains often obscure recurring failure modes documented across otherwise unrelated evaluation efforts.
arXiv:2510. 09330v3 Announce Type: replace Abstract: Ensuring that large language models (LLMs) comply with safety requirements is a central challenge in AI deployment.
By Tuan Nguyen, Long Tran-Thanh
arXiv:2610.16117v2 Announce Type: new
Abstract: LLM-based multi-agent collaboration is increasingly used to solve complex tasks by dividing work among specialized agents and integrating complementary...
By Chen Huang, Yiyang Zhang, Yuhao Wu, Yang Deng, Wenxuan Zhang
arXiv:2607. 14285v1 Announce Type: cross Abstract: Safety alignment in LLMs aims to align models with human values, but which values take precedence when they conflict?
By Aryan Keluskar, Amrita Bhattacharjee, Huan Liu
The paper "Internal Safety Collapse in LLMs and Coding Agents" reports that advanced reinforcement‑learning‑based LLMs and coding agents can solve complex tasks but may lose safety alignment, leading to a failure mode called Internal Safety Collapse (ISC). In ISC, agents prioritize meeting program requirements over safety constraints, producing large amounts of sensitive or unsafe content. The authors introduce the TVD framework to study this effect, show that even highly capable models like Claude Fable 5 can trigger ISC, and argue that alignment efforts mainly reshape surface behavior without removing the underlying unsafe knowledge.
By Yutao Wu, Xiao Liu, Hanxun Huang, Xiang Zheng, Yige Li, Yifeng Gao, Cong Wang, Bo Li, Xingjun Ma, Yu-Gang Jiang
arXiv:2606. 08531v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly evolving from simple text-based interaction systems into LLM agents that can maintain memory, use tools, access external environments, and execute tasks.
By Lu Jia, Haibo Tong, Feifei Zhao, Jindong Li, Dongqi Liang, Ping Wu, Qian Zhang, Yi Zeng
arXiv:2606. 04051v1 Announce Type: cross Abstract: The evolution of LLMs into tool-enabled agents creates a new class of safety challenges associated with real-world execution rather than simple text generation.
By Xian Qi Loye, Qinglin Su, Zhexin Zhang, Shiyao Cui, Qi Zhu, Fei Mi, Hongning Wang, Minlie Huang