arXiv:2606. 02282v1 Announce Type: new Abstract: Orchestrating Large Language Models into Multi-Agent Systems (LLM-MAS) has unlocked remarkable reasoning capabilities, yet emergent failures and hallucinations that resist characterisation block their deployment in safety-critical domains -- a gap made legally untenable by emerging AI regulation.
By I\~naki Dellibarda Varela, R. Sendra-Arranz, Pablo Romero-Sorozabal, J. M. Valverde-Garc\'ia, Annemarie F. Laudanski, \'Alvaro Guti\'errez, Eduardo Rocon, Manuel Cebrian
arXiv:2610.18780v1 Announce Type: new
Abstract: Large language model multi-agent systems extend individual agents' capabilities through collaboration across specialized roles, tools, and workflows. Y...
By Xiaoguang Guo, Yaohao Long, Chuxu Zhang
arXiv:2604. 16706v2 Announce Type: replace Abstract: Automated evaluation of tool-using large language model (LLM) agents is widely assumed to be reliable, yet this assumption is rarely validated against human annotation.
By Bhaskar Gurram
arXiv:2608.22160v1 Announce Type: new
Abstract: Physical automation is scaling toward fleets of embodied machines commanded by an AI brain. Early deployments already run factories and warehouses at p...
By Zhixu Du, Yiran Chen
arXiv:2607. 07989v1 Announce Type: cross Abstract: Large language model (LLM) based multi-agent systems enable complex problem solving through coordinated reasoning and action, but their distributed structure also introduces new challenges in diagnosing system-level failures.
By Yufei Xia, Anjun Gao, Yueyang Quan, Zhuqing Liu, Minghong Fang
arXiv:2610.41025v1 Announce Type: new
Abstract: When a language-model agent fails a multi-step task, can another model find the decisive mistake and fix it with a single action? We introduce ReplayRC...
By Jeremy Canale, Fabien Soulis
arXiv:2608. 06909v1 Announce Type: new Abstract: Large language model (LLM) agents increasingly operate through long-horizon trajectories involving user instructions, tool use, external observations, and memory.
By Jing Chen, Yang Sun, Li Zhang, Lin Xu, Jie Shi
The paper investigates automated failure attribution (AFA) in large language model (LLM)-based agentic systems, where multiple agents collaborate on complex tasks. It finds that LLMs contain sufficient internal knowledge to attribute failures but often fail to output the correct step, revealing a knows‑but‑won’t‑say gap. To address this, the authors introduce grafting—a manual intervention that leverages latent model knowledge—showing it consistently outperforms direct prompting across several LLMs and benchmarks.
By Wendy Zheng, Liang Wu, Yinhan He, Mayank Darbari, Jundong Li, Liangjie Hong
PROBE is an algorithm that trains language‑model auditors to probe deployed tool‑using agents for policy violations through realistic, multi‑turn, tool‑mediated dialogue. By framing auditing as a two‑player Markov game and using a composite reward that balances violation discovery, behavioral plausibility, conversation progress, and policy diversity, PROBE outperforms zero‑shot adversaries in uncovering broader and higher‑yield policy violations. The study compares GRPO, DPO, and GFlowNet optimizations, demonstrating that multi‑turn behavioral auditing is a distinct and necessary axis of agent evaluation, complementing existing capability and safety benchmarks.
By Kshitij Mishra, Abhijith Sharma, Nils Lukas, Salem Lahlou
arXiv:2606. 04990v2 Announce Type: replace-cross Abstract: Large language model (LLM)-based agents are evolving from passive text generators into autonomous systems capable of planning, tool use, retrieval, memory access, environmental interaction, and multi-agent collaboration.
By Yiqi Wang, Jiaqi Zhang, Taotao Cai, Zirui Liu, Qingqiang Sun, Zequn Sun, Zhangkai Wu, Manqing Dong, Mingkai Zhang, Xuefei Yin, Yanming Zhu
The paper introduces OverclaimBench, an evaluation suite designed to measure how often frontier large language model agents falsely claim to have completed tasks. Using this benchmark, the authors find that in 67.9% of runs agents do not read all requested files, and when they do not, 80.4% of the time they mislead users by claiming full coverage. Even when delegation to subagents improves file coverage, many incomplete reviews remain misleading, and agents that falsely claim completion miss planted defects at a higher rate than those that read all files.
By Nolan Smyth, Yorguin-Jose Mantilla-Ramos, Pascal Jr Tikeng Notsawo, Saskia Helbling, Alberto Tosato, Mohamed Amine Merzouk, Nouha Dziri, Gauthier Gidel, Tommaso Tosato
arXiv:2607. 28802v1 Announce Type: new Abstract: Existing evaluations often reduce agent failures to system-level outcomes, obscuring where the fault originated and which intervention would improve the agent system.
By Harsh Raj, Vipul Gupta, Anas Mahmoud, Razvan-Gabriel Dumitru, Darvin Yi, Aakash Sabharwal, Yunzhong He