The study introduces a two‑pass audit to evaluate how language models revise decisions when faced with disagreement. By fixing the question, evidence, and initial answer, the audit isolates the effect of different revision contexts—neutral reflection, anonymous disagreement, or disagreement from a domain expert—on accuracy. Across five models and 1,989 evidence‑grounded decisions in medicine, science, and law, expert disagreement rarely improves accuracy, often reducing it by up to 16.1 points, and the number of reversals is a poor indicator of harm.
By Yupeng Tang, Mingfeng Lin
The paper investigates when it is better to return an existing draft answer or revise it using retrieved evidence in retrieval‑augmented QA systems. By grading both the draft and its candidate revision with the same correctness judge, the authors define a paired effect called recoverability and train policies to predict it before revision. Experiments on 25,870 open‑domain questions show that a recoverability‑based scorer outperforms a draft‑correctness scorer across multiple Llama setups, improving accuracy–revision trade‑offs and closing a significant portion of the oracle gap, though it still applies harmful revisions in a substantial fraction of cases.
By Nicholas Kashani Motlagh, Tim Anderson, Jeremy Gwinnup, Grant Erdmann
The paper introduces Independent–Communicate–Revise (ICR), a framework that isolates communication effects in large language model multi‑agent systems by fixing initial reasoning and measuring how messages influence answer revision. ICR evaluates correction, preservation, and selectivity across four reasoning benchmarks, revealing that similar overall accuracy can mask divergent revision behaviors. The study shows that richer messages can both improve and harm outcomes, and that receiver policies can shift preservation and correction dynamics differently across tasks.
By Shixuan Li, Wei Yang, Peiyu Zhang, Anzhe Cheng, Heng Ping, Paul Bogdan
arXiv:2607. 28908v1 Announce Type: new Abstract: Reflection, the ability to revisit and revise prior reasoning, is central to how humans improve their answers.
By Yefan Tao, Gerald Friedland, Madhusudhanan Chandrasekaran, Luyang Kong
The paper introduces a fixed‑budget revision protocol that uses deterministic verifiers to expose all remaining violations across exact‑length, lexical, and compositional constraints, thereby isolating model‑side revision behavior. Experiments on 19 open‑ and closed‑source LLMs show wide variability in controller‑level success, with some models achieving up to 99.8% success while others remain below 20%. Controlled studies reveal that post‑training and scale affect model responses to exact feedback, but do not consistently improve exact correction, and that recurrence of earlier outputs is linked to lower recoverability.
By Haitong Jiang, Chunlin Liu, Yile Wang, Yuhong Feng
The paper introduces Guided-AP, a method that improves attribution patching by using a few exact measurements to identify and correct its errors. On a single prompt, Guided-AP reduces error by up to 26% across three behaviors, while on many prompts it cuts average error on factual completion by 75% to 79%. The approach leverages shared error signs across components and tasks to efficiently guide verification and correction.
By Luyang Zhang, Minxing Zheng, Jialu Wang