arXiv:2609.38324v1 Announce Type: cross
Abstract: Multi-agent systems of LLMs add discussion to majority voting and are therefore expected to be more capable. However, empirical reports conflict on w...
By Chand Sahil Mansuri, Xin Wang, Mengying Li, Bryan Acton, Rory Eckardt, Dhaval Patel, Sadamori Kojaku
The paper identifies a "consensus trap" where majority voting among large language model agents can reinforce incorrect answers when the majority follows misleading instructions. It proposes chunk‑interleaved Round‑Robin (RR) collaboration, allowing agents to take turns extending a shared reasoning trajectory so that minority agents can insert corrections before the final answer is produced. Experiments across model families and reasoning benchmarks show that RR improves accuracy under majority corruption while maintaining performance under minority corruption, and that shared reasoning during turns tends to increase support for the correct answer.
By Jiayuan Liu, Shiyi Du, Weihua Du, Xinhao Cheng, Mingyu Guo, Vincent Conitzer
arXiv:2604. 07667v2 Announce Type: replace Abstract: Multi-agent debate improves LLM reasoning, yet agreement is not evidence of correctness.
By Mengdie Flora Wang, Haochen Xie, Guanghui Wang, Aijing Gao, Guang Yang, Ziyuan Li, Qucy Wei Qiu, Fangwei Han, Hengzhi Qiu, Yajing Huang, Bing Zhu, Jae Oh Woo
Large language model (LLM) agents now fix software, operate computers and serve customers, but their harnesses, the programs that run them, call a large LLM at every step, which makes long tasks slow...
The paper introduces Bayesian Dialectical Argumentation (BDA), a method for aggregating answers from multiple large language models (LLMs) in a council setting. BDA treats each LLM’s typed moves—proposals, challenges, and concessions—as evidence in a classical annotator model, estimating per-agent reliability even when some agents are persistently unreliable. By weighting evidence according to these inferred reliabilities, BDA produces calibrated posterior probabilities for candidate answers and can invert unreliable agents instead of merely outvoting them, achieving superior calibration and robustness on both binary and multi-class benchmarks without extra LLM calls.
By Ionel Eduard Stan, Paolo Napoletano
TeamGym is a new environment that evaluates and trains language‑model agents on team decision‑making tasks drawn from real‑world legal, finance, medical, and software engineering texts. It introduces the concept of pivotal information—a minimal set of facts necessary for a correct decision—and tracks how agents share and retrieve this information. Experiments show that distributing pivotal facts and requiring agents to extract them independently significantly reduces success rates, but reinforcement‑learning training can improve decision quality and foster coordination among team members.
By Chanwoo Park, Jaywoong Jeong, Zachary Wojtowicz, Yubin Kim, Taekyun Lee, Kaiqing Zhang, Asuman Ozdaglar
arXiv:2608. 09324v1 Announce Type: new Abstract: On unlabeled test data, reinforcement learning lacks a ground-truth reward; test-time RL methods derive one from the model's own roll-outs, rewarding those that match the majority vote over $N$ sampled answers.
By Ambuj Mehrish, Sebastiano Vascon
Seq@K is a new evaluation protocol that measures a language model’s ability to solve a problem within K sequential attempts, receiving feedback after each failure. It isolates the core capability of iterative refinement from feedback, contrasting with existing metrics like Pass@K that do not capture this behavior. Experiments on seven benchmarks and sixteen frontier models show that Seq@K can change model rankings, reveal more efficient configurations, and provide insights into how models use feedback.
By Thomson Yen, Gilbert Yang, Naimeng Ye, Minghui Chen, Dong Xia, Tianyi Shen, Yechan Jeon, Yaqin Chen, Daksh Mittal, Fangyu Wu, Hongseok Namkoong
The paper investigates why reinforcement learning with self‑generated labels—specifically majority‑vote rewards—can recover most of the performance gains of training with verifiable rewards. It argues that the key factor is batch‑level updates: a prompt’s own gradient contributes less than 1% to its output change, so only coherent label errors across prompts influence learning. Experiments on five models show that majority‑vote updates align almost perfectly with gold updates, with misdirection disappearing as batch size grows and incoherent errors behaving like random noise.
By Dylan P. Wilson, Chengsong Huang, Jiaxin Huang
arXiv:2511.11500v3 Announce Type: replace
Abstract: Modern language models fail a fundamental requirement of trustworthy intelligence: knowing when not to answer. Despite achieving impressive accurac...
By Mohamad Amin Mohamadi, Tianhao Wang, Zhiyuan Li
arXiv:2611.17563v1 Announce Type: cross
Abstract: A benchmark usually measures a language model alone with a question, and deployment rarely looks like that: answers get contested, and models give up...
By Soheil Abbasloo, Peng Cheng
arXiv:2610.17869v1 Announce Type: new
Abstract: Multi-agent Large Language Model (LLM) systems typically commit to a deliberation protocol before inference. A fixed protocol cannot allocate computati...
By Yuan Liu, Haoyue Peng, Fanliang Hu, Lei Huang, Wenli Cai