The study examines how shared item‑level error limits the benefit of aggregating small language models in Fermi estimation tasks. By decomposing log10 errors into item‑specific and residual components across 2.51 M answers from 15 open‑weight models, it finds that item‑level error accounts for about 91 % of variance, reducing the expected error gain from a 30‑model team to only 1.7 % (max 5.4 %) instead of the theoretical 74‑79 %. Techniques such as three rounds of debate, model mixing, and temperature adjustments yield modest improvements, but no method surpasses the most accurate single model.
The paper introduces Steiner’s taxonomy of group tasks to study how multi‑agent large language model (LLM) teams scale on disjunctive versus compensatory tasks. By modeling agents as conditionally independent given the item, it shows that plurality voting converges to the modal answer while averaging converges to the item‑level bias. Experiments with 13 open‑weight models and up to 30 agents reveal that disjunctive tasks benefit from larger teams, whereas compensatory tasks like Fermi estimation see little improvement, highlighting that task structure and aggregation method fundamentally determine team scaling.
By Carolina Fortuna, Blaz Bertalanic
The study evaluates multi‑agent debate (MAD) in small language models, testing whether cognitive diversity—via personas, sampling temperature, or model identity—drives performance gains. Across 23 models, five tasks, and over 5,500 runs, MAD consistently outperforms single‑model inference but, when matched for generation budget, it ties or falls behind self‑consistency sampling, with persona prompting actually reducing accuracy. The authors find that MAD’s benefits largely stem from the first answer exchange and that many reported gains are due to ensemble‑sampling effects rather than true diversity, highlighting the need for budget‑matched, contamination‑checked baselines.
whyItMatters":"The findings clarify that MAD’s perceived advantages may be overestimated and that future debate mechanisms must be evaluated against rigorous, budget‑matched baselines to ensure genuine performance improvements."
By Leonardo Ferreira, Gardenia Liu, Kaden Zheng
arXiv:2607. 18310v1 Announce Type: cross Abstract: Synthetic-population tools increasingly run every individual as an independent large language model (LLM) agent.
By Gurkan Ozkan
The study compares single-agent and multi-agent large language model (LLM) setups for reviewing credit assessment memos against specified requirements. In a controlled experiment with 300 synthetic memos, a single agent examined six requirements in one call, while a team of three agents each checked two requirements. Results show the team identified 0.08 to 0.10 more defects on certain models but also produced more false alarms, used 2.2 to 2.4 times more tokens, and had mixed effects on balanced accuracy depending on instruction and model.
By Pei Liu
The study compares human deliberation in Wason selection tasks with large language model (LLM) agent groups that are seeded with participants’ pre-discussion beliefs. Across various scoring definitions, human consensus rates ranged from 24.0% to 57.0%, whereas LLM agents consistently achieved higher consensus, with gaps of 34–44 percentage points in two sensitivity analyses. Even when early stopping was removed or memorizable answers were eliminated, LLM groups still reached near-unanimous agreement, often on incorrect answers, indicating that simulated consensus does not reflect collective accuracy.
By Tengfei Shao
The study evaluates six frontier language models on a two‑agent <log(N)>‑Questions game using Wikipedia lead paragraphs. In each game a questioner must identify a target paragraph with exactly <log2 N> yes/no questions, while an answerer only sees the target and the question and replies with a single word. Across 408 games, the models perform similarly, with Claude Opus 5 winning 28 of 68 games and the top five models showing only marginal differences; win rates decline sharply with larger document sets, following a reliability parameter of 0.928 per question.
"whyItMatters":"The results reveal how well language models can communicate under information asymmetry, highlighting that even top models struggle to extract a full bit per question and that reasoning token usage does not strongly predict success."
By Peter Potash
arXiv:2608. 15980v1 Announce Type: cross Abstract: Preference benchmarks are built by hiring annotators, and the identity of those annotators is treated as an implementation detail.
By Anik Jha
The study investigates how limited reading capacity and claim wording influence consensus outcomes in language‑model networks. By modeling message capacity as the number of messages an agent reads, the authors show that when agents read fewer than about 6.4 messages on average, a wrong consensus becomes unreachable. However, the wording of a claim—its inherent threshold—can override this effect, leading to incorrect consensus even when most agents start correct.
By Makoto Fukushima
The paper introduces TalkMesh, a decentralized network of small language model agents that learn to communicate effectively during inference. Each agent proposes an answer, scores it with a confidence head, and the most confident agent broadcasts a hint; lower‑confidence agents revise their proposals if a new suggestion scores higher. This gossip‑based consensus, trained via group relative policy optimization, enables a mesh of three agents to match the accuracy of majority voting over 32 samples, and scales to larger meshes to significantly boost performance on benchmarks like GSM8K and MATH-500.
By Mehmet Kerem Turkcan
The paper evaluates multi‑agent language models (LLMs) by comparing them to single‑pass baselines across nine empirical studies. It finds that multi‑agent systems generally do not outperform single‑pass baselines, with negative or near‑zero accuracy differences when matched on sample count or resource budget. The authors conclude that communication alone offers no advantage unless agents have partitioned evidence, external verification, or diverse errors, and they provide a reviewer checklist and accounting tools.
By Seyed Mahmoud Sajjadi Mohammadabadi
arXiv:2607. 10202v1 Announce Type: new Abstract: Cross-model comparisons read divergence in value dispositions as evidence that language models hold individuated values.
By Hong-In Won, Jinseok Jang, Hyoseop Kim