The study examines how shared item‑level error limits the benefit of aggregating small language models in Fermi estimation tasks. By decomposing log10 errors into item‑specific and residual components across 2.51 M answers from 15 open‑weight models, it finds that item‑level error accounts for about 91 % of variance, reducing the expected error gain from a 30‑model team to only 1.7 % (max 5.4 %) instead of the theoretical 74‑79 %. Techniques such as three rounds of debate, model mixing, and temperature adjustments yield modest improvements, but no method surpasses the most accurate single model.
The paper introduces Steiner’s taxonomy of group tasks to study how multi‑agent large language model (LLM) teams scale on disjunctive versus compensatory tasks. By modeling agents as conditionally independent given the item, it shows that plurality voting converges to the modal answer while averaging converges to the item‑level bias. Experiments with 13 open‑weight models and up to 30 agents reveal that disjunctive tasks benefit from larger teams, whereas compensatory tasks like Fermi estimation see little improvement, highlighting that task structure and aggregation method fundamentally determine team scaling.
By Carolina Fortuna, Blaz Bertalanic
The study evaluates multi‑agent debate (MAD) in small language models, testing whether cognitive diversity—via personas, sampling temperature, or model identity—drives performance gains. Across 23 models, five tasks, and over 5,500 runs, MAD consistently outperforms single‑model inference but, when matched for generation budget, it ties or falls behind self‑consistency sampling, with persona prompting actually reducing accuracy. The authors find that MAD’s benefits largely stem from the first answer exchange and that many reported gains are due to ensemble‑sampling effects rather than true diversity, highlighting the need for budget‑matched, contamination‑checked baselines.
whyItMatters":"The findings clarify that MAD’s perceived advantages may be overestimated and that future debate mechanisms must be evaluated against rigorous, budget‑matched baselines to ensure genuine performance improvements."
By Leonardo Ferreira, Gardenia Liu, Kaden Zheng
arXiv:2607. 18310v1 Announce Type: cross Abstract: Synthetic-population tools increasingly run every individual as an independent large language model (LLM) agent.
By Gurkan Ozkan
The study compares single-agent and multi-agent large language model (LLM) setups for reviewing credit assessment memos against specified requirements. In a controlled experiment with 300 synthetic memos, a single agent examined six requirements in one call, while a team of three agents each checked two requirements. Results show the team identified 0.08 to 0.10 more defects on certain models but also produced more false alarms, used 2.2 to 2.4 times more tokens, and had mixed effects on balanced accuracy depending on instruction and model.
By Pei Liu
The study compares human deliberation in Wason selection tasks with large language model (LLM) agent groups that are seeded with participants’ pre-discussion beliefs. Across various scoring definitions, human consensus rates ranged from 24.0% to 57.0%, whereas LLM agents consistently achieved higher consensus, with gaps of 34–44 percentage points in two sensitivity analyses. Even when early stopping was removed or memorizable answers were eliminated, LLM groups still reached near-unanimous agreement, often on incorrect answers, indicating that simulated consensus does not reflect collective accuracy.
By Tengfei Shao