← Back to all news
arXiv AI By Chenhao Wu, Haoxuan Jia, Yang Liu, Yingguang Yang, Yuhan Lin, Chongyang Zhang, Hao Zheng, Yulin Huang, Jianshen Zhang, Yongzhi Qi, Shang Luo, Kefu Xu, Jifeng Zhu, Bin Chong

Safety Does Not Compose: Non-Decaying Loop State for Autonomous LLM Agents

Read the original on arXiv AI →

The paper demonstrates that safety mechanisms for autonomous large language model agents fail to compose across iterative loops, as trajectory‑scoped monitors cannot detect attacks whose evidence is spread over multiple iterations. It introduces LoopHarness, a system that maintains a persistent, non‑decaying safety state across loops, bounding unauthorized actions with a constant that does not grow with the number of iterations. The authors provide a comprehensive evaluation protocol, including attacks that require cross‑iteration evidence, module ablations, and adaptive white‑box red‑team testing.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Jul 29

Early Detection of Distributed Backdoors in Multi-Agent LLM Systems: A Characterization Study

arXiv:2607. 24893v1 Announce Type: cross Abstract: Multi-agent LLM systems can be attacked by a payload that no single agent ever holds in full: a poisoned tool hides encrypted fragments in its observations, spreads them across several agents, and an external step reassembles and executes them after the run.

By Diego Fernandez Arias, Dev Prashant Mistry, Ren Wang, Yibo Hu
llmsagentsfine-tuningsafety
More like this →
arXiv AI
Jun 19

LLM agent safety, multi-turn red-teaming, jailbreak benchmarks, adversarial robustness, safety-critical systems

arXiv:2606. 20408v1 Announce Type: cross Abstract: Large language model (LLM) agents are increasingly proposed as supervisory components for safety-critical systems, yet their robustness under sustained, adaptive adversarial pressure remains poorly characterized.

By Hanwool Lee, Dasol Choi, Bokyeong Kim, Seung Geun Kim, Haon Park
llmsagentsbenchmarkssafety
More like this →
arXiv AI
Jul 7

NRT-Bench: Benchmarking Multi-Turn Red-Teaming of LLM Operator Agents in Safety-Critical Control Rooms

arXiv:2606. 20408v3 Announce Type: replace-cross Abstract: Large language model (LLM) agents are increasingly proposed as supervisory components for safety-critical systems, yet their robustness under sustained, adaptive adversarial pressure remains poorly characterized.

By Hanwool Lee, Dasol Choi, Bokyeong Kim, Haon Park, Seung Geun Kim
llmsagentsbenchmarkssafety
More like this →
arXiv Computation and Language
Aug 28

The Cold-Start Safety Gap in LLM Agents

The paper investigates whether tool‑calling large language model agents maintain consistent safety throughout a conversation. It finds that agents are most vulnerable at the very start of a session, with safety improving significantly after completing a few regular agentic tasks—a phenomenon termed the cold‑start safety gap. The authors introduce the Safety Over Depth for Agents (SODA) benchmark to systematically study this effect, evaluate multiple models, and demonstrate that warming up agents with regular tasks before deployment enhances safety while preserving utility.

By Chung-En Sun, Linbo Liu, Tsui-Wei Weng
llmsagentsbenchmarkssafety
More like this →