The paper argues that accuracy is an inadequate metric for certifying a coordinator’s theory‑of‑mind in multi‑agent systems, because the required accuracy depends on deployment‑specific costs and errors are often dependent. Experiments on task assignment, Level‑Based Foraging, Hanabi, and human judgments show that accuracy can both approve harmful language‑model deployments and reject beneficial ones. The authors propose a Blind‑Spot Gate that evaluates error costs, pairwise interactions, and dependence without deployment, achieving better safety outcomes than accuracy alone.
By Will Beaumaster, Paul Schrater
Coordinators that act on what they infer about teammates are usually certified by the accuracy of those inferences. We argue that accuracy is the wrong certificate, for two reasons. First, the accurac...
The study investigates how humans and AI collaborate on a puzzle task, focusing on referential uncertainty—when a description could refer to multiple objects. It finds that eliciting a belief distribution over candidate pieces yields better calibration and discrimination than raw action probabilities, and that precise descriptions or well‑targeted hedges significantly reduce the acceptance of wrong placements. However, the AI rarely externalizes uncertainty, and poorly targeted hedges can be counterproductive.
By Christian Poelitz, Finale Doshi-Velez, Si\^an Lindley
AgentAudit is an open, extensible framework that evaluates the full lifecycle of AI agents, assessing planning, tool selection, execution, memory, and reasoning across ten dimensions such as instruction integrity, security, and alignment. Unlike existing benchmarks that focus on single aspects, AgentAudit analyzes the entire execution trace to attribute failures to specific stages. The framework was tested on five large language models, revealing significant differences in trustworthiness even among models with similar task‑completion performance.
By Shrey Nag, Sachita, Abhishek Kumar Singh, Lipi Goel, Rajeshwar Singh Janwar
arXiv:2610.14530v1 Announce Type: cross
Abstract: Multi-robot systems (MRS) offer significant advantages over single-robot platforms by enabling information sharing and coordination for different tas...
By Ruiyang Wang, Hao-Lun Hsu, Haocheng Meng, Songqiao Xie, Miroslav Pajic
arXiv:2608. 08601v1 Announce Type: new Abstract: To anticipate socio-technical risks from AI agents, organizations need taxonomies to classify them.
By Gabriele La Malfa, Lakmal Meegahapola, Edyta Bogucka, Jie M. Zhang, Michael Luck, Elizabeth Black, Daniele Quercia