The paper argues that accuracy is an inadequate metric for certifying a coordinator’s theory‑of‑mind in multi‑agent systems, because the required accuracy depends on deployment‑specific costs and errors are often dependent. Experiments on task assignment, Level‑Based Foraging, Hanabi, and human judgments show that accuracy can both approve harmful language‑model deployments and reject beneficial ones. The authors propose a Blind‑Spot Gate that evaluates error costs, pairwise interactions, and dependence without deployment, achieving better safety outcomes than accuracy alone.
By Will Beaumaster, Paul Schrater
Coordinators that act on what they infer about teammates are usually certified by the accuracy of those inferences. We argue that accuracy is the wrong certificate, for two reasons. First, the accurac...
The study investigates how humans and AI collaborate on a puzzle task, focusing on referential uncertainty—when a description could refer to multiple objects. It finds that eliciting a belief distribution over candidate pieces yields better calibration and discrimination than raw action probabilities, and that precise descriptions or well‑targeted hedges significantly reduce the acceptance of wrong placements. However, the AI rarely externalizes uncertainty, and poorly targeted hedges can be counterproductive.
By Christian Poelitz, Finale Doshi-Velez, Si\^an Lindley
AgentAudit is an open, extensible framework that evaluates the full lifecycle of AI agents, assessing planning, tool selection, execution, memory, and reasoning across ten dimensions such as instruction integrity, security, and alignment. Unlike existing benchmarks that focus on single aspects, AgentAudit analyzes the entire execution trace to attribute failures to specific stages. The framework was tested on five large language models, revealing significant differences in trustworthiness even among models with similar task‑completion performance.
By Shrey Nag, Sachita, Abhishek Kumar Singh, Lipi Goel, Rajeshwar Singh Janwar
arXiv:2610.14530v1 Announce Type: cross
Abstract: Multi-robot systems (MRS) offer significant advantages over single-robot platforms by enabling information sharing and coordination for different tas...
By Ruiyang Wang, Hao-Lun Hsu, Haocheng Meng, Songqiao Xie, Miroslav Pajic
arXiv:2608. 08601v1 Announce Type: new Abstract: To anticipate socio-technical risks from AI agents, organizations need taxonomies to classify them.
By Gabriele La Malfa, Lakmal Meegahapola, Edyta Bogucka, Jie M. Zhang, Michael Luck, Elizabeth Black, Daniele Quercia
arXiv:2610.27965v1 Announce Type: new
Abstract: Can humans and artificial intelligence cooperate such that their combination yields greater accuracy than either agent alone? This result, termed \emph...
By Andrea Campagner, Federico Cabitza, Vincenzo Crupi
The paper introduces Traverse, a benchmark of 2,518 agent trajectories and 6,967 annotated mistakes across software engineering, computer use, and science tasks, revealing that failures often go unrecovered and can cause irreversible harm before a run is deemed successful. It shows that human judges struggle to detect the first mistake in most runs, while a 4‑billion‑parameter verifier called Scout can locate failures more effectively and improve task success when used to select among candidate runs. The study demonstrates that making failure detection inexpensive and reliable can enable long‑horizon agents to learn from their own mistakes and increase trustworthiness in autonomous AI.
By Salman Rahman, Yubin Kim, Mihir Parmar, A. Ali Heydari, Genglin Liu, Simon A. Lee, Weizhi Zhang, Arian Hosseini, Ahmed A. Metwally, Yuzhe Yang, Baharan Mirzasoleiman, Xin Liu, Pavel Izmailov, Saadia Gabriel, Mark Malhotra, Shwetak Patel, Daniel McDuff, Hamid Palangi
The study examines how the speed and accuracy of an AI teammate—Fast/Less-Accurate (FLA-AI) versus Slow/Accurate (SA-AI)—affect performance in a collaborative Brain‑Computer Interface (cBCI) team during a virtual reality drone search task. Fast AI leads to instant, blind compliance and a sharp drop in human accuracy, while Slow AI induces delayed cognitive conflict that ultimately allows teams to recover and achieve perfect accuracy. A 2D Adaptive Riemannian Oracle and Hybrid Fusion techniques were used to adaptively capture and integrate these timing-dependent signals, improving team performance in both scenarios.
By Christopher Baker, Stephen Hinton, Akashdeep Nijjar, Riccardo Poli, Caterina Cinel, Tom Reed, Stephen Fairclough
arXiv:2610.15136v1 Announce Type: new
Abstract: Trust calibration remains the dominant framing of AI-assisted decision making: the goal is to close the gap between perceived and actual trustworthines...
By Michaela Benk, Kacper Sokol
The paper introduces a partition theory of AI evaluation, arguing that an evaluator’s ability to certify a model’s capabilities is limited by its perception. It shows that if a model’s input partition refines the evaluator’s, it can pass all tests the evaluator can grade, but only a stricter evaluator can raise the discrimination floor. The authors define an echo test that is complete for the evaluator’s tests, proving that passing it only guarantees a minimum discrimination level and cannot detect exact imitation of weaker models.
By Ahmed M. Mahran
arXiv:2606. 14200v1 Announce Type: new Abstract: Open platforms increasingly route tasks among heterogeneous LLM agents--differing in base model, scaffold, and tool stack--whose competence varies sharply by skill: an agent excellent at one skill may be useless at another.
By Yihan Xia, Taotao Wang