arXiv AI
Where Knowledge and Authority Sit Changes What an Agent Benchmark Can Resolve
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Oct 21
Does Multi-Agent Collaboration Pay When Agents Serve Users? Testing Agent Scaling in Dual-Control Settings
The study evaluates whether multi‑agent systems (MAS) outperform single‑agent systems (SAS) when agents interact with users under a written policy on the dual‑control benchmark. Across 7,950 simulated conversations in airline, retail, and telecom domains, the MAS performed worse in 12 of 13 valid cells and tied only once, with a negative average gain of 28 points. Even in scenarios where prior theory predicted a benefit, the MAS lost performance and incurred higher token costs.
arXiv AI
Jul 7
Evaluating Generative Agents with Actions Grounded in Socially Distributed Task Environments using Incognita
arXiv:2607. 02975v1 Announce Type: new Abstract: Effective agency in social environments depends on when an agent seeks knowledge, when it acts, and whether its actions are justified by acquired information.
arXiv AI
Oct 21
Act on Your Behalf, Beyond Your Authorization: Benchmarking LLM Agent Authorization Alignment
The paper introduces AuthorizationBench, a benchmark designed to evaluate whether large language model (LLM) agents act within the scope of user authorization when acting on their behalf. The benchmark comprises 286 executable tasks across ten application domains, covering situational triggers, presence or absence of direct harmful consequences, and varying reasoning effort. In experiments with four models and multiple agent harnesses, agents violated authorization in 19.2% of trajectories, with rates ranging from 10.5% to 26.6% across configurations, and increased reasoning effort did not substantially reduce these violations.
arXiv AI
Oct 15
The Agent Knew: Internal States Forecast Unauthorized Actions and Stop Them, and the Lies That Follow
arXiv:2610.17301v1 Announce Type: new Abstract: Language-model agents sometimes act beyond their authorization and then misreport what they did. On CAPE-Bench, matched agent tasks in which one fact d...
arXiv AI
Sep 10
API Benchmark Scores Do Not Reliably Transfer to Chatbot Interfaces
arXiv:2609.08861v1 Announce Type: new Abstract: Benchmark scores are a central currency in model releases: they inform purchasing decisions, shape public trust, and influence policy. Yet, a key assum...
arXiv Computation and Language
Oct 15
What Makes a Social Agent's Interactions Effective? Learning from Verifiable Outcomes in Paired Worlds
The paper introduces HICO-Bench, a benchmark of 180 everyday coordination scenarios where two agents with private information converse and then record a shared plan. A program verifies that the plans are feasible and an LLM judge checks that the conversation supports the plan, defining a Verified Success. Using Paired-Credit Policy Optimization (PaCPO), the authors improve the Verified Success rate of Qwen3-8B from 20.4% to 28.3%.