arXiv:2604. 07667v2 Announce Type: replace Abstract: Multi-agent debate improves LLM reasoning, yet agreement is not evidence of correctness.
arXiv:2609.38324v1 Announce Type: cross Abstract: Multi-agent systems of LLMs add discussion to majority voting and are therefore expected to be more capable. However, empirical reports conflict on w...
The paper studies how to safely delegate action approval to multiple AI reviewers when the reviewers themselves may be misaligned. It introduces a weaker condition—k‑robust coalitional alignment—under which a threshold rule that tolerates up to k disapprovals guarantees that the principal’s expected utility is at least as good as a baseline policy. The authors extend this characterization to sequential decision‑making in discounted MDPs and show that full‑panel coverage of reward functions ensures safety in Nash equilibria, while more permissive thresholds can lead to unsafe outcomes. Experiments demonstrate that collective review can remain sound even when individual reviewers are not fully aligned, provided some disapprovals are allowed.
The paper introduces TalkMesh, a decentralized network of small language model agents that learn to communicate effectively during inference. Each agent proposes an answer, scores it with a confidence head, and the most confident agent broadcasts a hint; lower‑confidence agents revise their proposals if a new suggestion scores higher. This gossip‑based consensus, trained via group relative policy optimization, enables a mesh of three agents to match the accuracy of majority voting over 32 samples, and scales to larger meshes to significantly boost performance on benchmarks like GSM8K and MATH-500.
arXiv:2609.08016v1 Announce Type: new Abstract: Multi-agent debate, in which several LLMs exchange arguments before answering, is widely assumed to improve answer quality by surfacing genuine disagre...
arXiv:2606. 19111v1 Announce Type: cross Abstract: Team science holds that leadership is contingent: it helps only under specific conditions, and capable, autonomous teams may need none at all.
ECMR: Evidence-Aware Deliberation Control for Multi-Agent LLM Reasoning proposes a controller that dynamically manages deliberation steps in multi-agent LLM systems. The controller evaluates answer concentration, support diversity, contradictions, role behavior, verification history, and remaining budget to decide actions such as propose, challenge, verify, arbitrate, diversify, correct, or stop. In a held‑out evaluation on MedQA, CaseHOLD, and FinQA, ECMR‑Full improves pooled accuracy from 62.1% to 84.9%, lowers false‑consensus from 40.7% to 17.0%, and raises evidence‑support from 58.0% to 88.7%.
arXiv:2610.17869v1 Announce Type: new Abstract: Multi-agent Large Language Model (LLM) systems typically commit to a deliberation protocol before inference. A fixed protocol cannot allocate computati...
The paper introduces multi-expert Conformal Risk Control (CRC) algorithms for pairwise LLM-as-a-Judge evaluation in open-ended dialogue. Two initial methods—Score Averaging and Decision Voting—aggregate at the score and decision levels, respectively, and outperform single-expert approaches on homogeneous expert panels. To address limited coverage on heterogeneous panels, the authors propose Marginal‑Calibrated Conformal Consensus (MC3), which captures distinct per‑expert scoring scales through threshold ratios while maintaining a unified decision function, and demonstrate its effectiveness on a new 1,800‑pair human pairwise‑preference benchmark called Panel.
arXiv:2608. 09324v1 Announce Type: new Abstract: On unlabeled test data, reinforcement learning lacks a ground-truth reward; test-time RL methods derive one from the model's own roll-outs, rewarding those that match the majority vote over $N$ sampled answers.
arXiv:2606. 19494v1 Announce Type: new Abstract: Multi-agent LLM deliberation, where agents exchange and revise answers over several rounds, is increasingly used to improve reasoning and accuracy, yet how and why it works is rarely modelled.
The paper introduces Bayesian Dialectical Argumentation (BDA), a method for aggregating answers from multiple large language models (LLMs) in a council setting. BDA treats each LLM’s typed moves—proposals, challenges, and concessions—as evidence in a classical annotator model, estimating per-agent reliability even when some agents are persistently unreliable. By weighting evidence according to these inferred reliabilities, BDA produces calibrated posterior probabilities for candidate answers and can invert unreliable agents instead of merely outvoting them, achieving superior calibration and robustness on both binary and multi-class benchmarks without extra LLM calls.
Consilience is an inference‑time orchestration framework that steers and certifies communication among multi‑agent large language models in hidden‑profile settings. It summarizes each discussion turn with a compact state of uncertainty, disagreement, evidence gain, redundancy, and premature consensus, then selects a communication intervention (challenge, clarify, seek evidence, or route) and speaker. A round‑wise conformal calibration procedure guarantees that the controller’s proposed action has bounded one‑step regret with high probability, and an acceptance mechanism enforces this guarantee for the executed action. Experiments on HiddenBench‑style tasks show that Consilience improves decision accuracy and communication efficiency over fixed and unstructured protocols, sometimes outperforming a full‑information baseline.