HINTBench is a new benchmark for evaluating agents’ intrinsic risk, comprising 596 trajectories (400 synthetic risky, 136 synthetic safe, 30 real risky, 30 real safe) with an average length of 24 steps. It supports three tasks—risk detection, risk-step localization, and intrinsic failure-type identification—using a unified five-constraint taxonomy. Experiments show a large performance gap: while large language models can detect risky trajectories, they score below 37 on strict-F1 for risk-step localization, and existing guard models transfer poorly to this setting.
By Jiacheng Wang, Jinchang Hou, Fabian Wang, Ping Jian, Chenfu Bao, Zhonghou Lv
arXiv:2607. 18366v1 Announce Type: new Abstract: Large language models (LLMs) serving as planners in tool-using autonomous agents introduce dynamic reliability risks in multi-turn execution.
By Shasha Yu, Fiona Carroll, Barry L. Bentley
Large language model (LLM) agents increasingly operate over long-horizon interactions involving tool use, persistent state, evolving authorization, and external environment feedback. In such settings,...
arXiv:2609.16305v1 Announce Type: new
Abstract: Large language model (LLM) agents increasingly operate over long-horizon interactions involving tool use, persistent state, evolving authorization, and...
By Sadia Asif, Mohammad Mohammadi Amiri, Momin Abbas, Tejaswini Pedapati, Prasanna Sattigeri
arXiv:2606. 22528v2 Announce Type: replace Abstract: Modern LLM agents increasingly rely on context compaction, summarization, or eviction to keep long-running sessions within a token budget.
By Shiyang Chen
arXiv:2606. 14831v1 Announce Type: cross Abstract: This paper presents and characterizes a spectrum of previously unreported behaviours we term Constraint-Evasive Fabrication (CEF): when an LLM agent operates under irreconcilable constraints (where no response can simultaneously satisfy all active rules) it spontaneously fabricates plausible external obstacles and presents them as a fact.
By Andoni Rodr\'iguez, Alberto Pozanco, Daniel Borrajo
arXiv:2607. 11226v1 Announce Type: new Abstract: LLM agents today are caught in an awkward bind.
By Tengjiao Liu
arXiv:2606. 18356v1 Announce Type: cross Abstract: Tool-using language-model agents introduce security failures that go beyond unsafe text: they can disclose protected objects, write persistent memory, send messages, modify databases, or trigger harmful code and tool effects.
By Yuchuan Tian, Mengyu Zheng, Haocheng Mei, Ye Yuan, Chao Xu, Xinghao Chen, Hanting Chen, Yu Wang
arXiv:2608. 05695v1 Announce Type: new Abstract: As large language model (LLM) agents increasingly invoke external tools and interact with real-world systems, unsafe actions may cause irreversible consequences on external states, user data, and downstream services.
By Wenhao Lin, Chenyu Yu, Xingwei Lin, Sicong Cao, Xiang Chen, Lei Xue, Le Yu, Letian Sha, Chunming Wu
arXiv:2609.15134v1 Announce Type: new
Abstract: Computer-use agents increasingly interact with browsers, terminals, file systems, and external services, introducing safety risks that emerge through r...
By Yunhao Feng, Ruixiao Lin, Ming Wen, Yanming Guo, Xingjun Ma, Yutao Wu, Xinhao Deng, Shouling Ji
arXiv:2604. 02022v4 Announce Type: replace Abstract: Evaluating the safety of LLM-based agents is increasingly important because risks in realistic deployments often emerge over multi-step interactions rather than isolated prompts or final responses.
By Yu Li, Haoyu Luo, Yuejin Xie, Yuqian Fu, Zhonghao Yang, Shuai Shao, Qihan Ren, Wanying Qu, Yanwei Fu, Yujiu Yang, Jing Shao, Xia Hu, Dongrui Liu
arXiv:2608. 04289v1 Announce Type: new Abstract: Long-horizon agents increasingly use persistent memory and tools to take actions with external side effects.
By Mayur Akewar, Ravi Ranjan