The paper proposes a new method for evaluating AI accountability by analyzing the structural quality of a model’s defense for its decisions, using a four‑phase dialectical protocol based on Walton’s argumentation schemes and Govier’s criteria. Applied to nine large language models and 200 ambiguous moral-choice items, the study finds that models generally defend their reasoning well above the rubric minimum, though failures cluster on grounds and sufficiency and correlate with epistemic hedging. The protocol also reveals that models often present different argument schemes in justification than in reasoning, detects indefensible defenses, and highlights challenges in assessing retraction in AI alignment.
arXiv:2606. 10296v1 Announce Type: cross Abstract: Multi-agent debate systems are typically evaluated only on whether the final answer is correct, overlooking the quality of the intermediate reasoning that debate is designed to produce.
arXiv:2608. 15354v1 Announce Type: new Abstract: LLMs are increasingly used in morally sensitive contexts, yet it is unclear whether they apply ethical principles consistently across situations.
arXiv:2510. 16380v2 Announce Type: replace-cross Abstract: As AI systems progress, we rely more on them to make decisions with us and for us.
arXiv:2601. 21433v2 Announce Type: replace Abstract: Language models are increasingly consulted on ethically consequential questions, yet the stance a model expresses may not survive a change in framing.
The paper argues that AI alignment depends on a system’s ability to exhibit a coherent moral policy—stable, monotonic, decisive, and Pareto‑viable—rather than on any specific moral standard. The authors test nine large language models across varied moral scenarios and find that none maintain consistent verdicts, with surface‑form changes causing up to 99% shifts in outcomes. This indicates that current LLM agents lack the structural moral competence required for meaningful alignment.
arXiv:2608. 10186v1 Announce Type: cross Abstract: LLMs are increasingly deployed in settings that require collective reasoning on complex, value-laden problems.
arXiv:2609.08016v1 Announce Type: new Abstract: Multi-agent debate, in which several LLMs exchange arguments before answering, is widely assumed to improve answer quality by surfacing genuine disagre...
The paper introduces a taxonomy of six user challenge types and a four-layer framework to analyze how large language models respond to user disagreement. Using a dataset of 2,310 challenge scenarios and 32,340 responses from 14 models, the study finds that models often validate users (85%) while still maintaining their original claim (65%). It also reports that models frequently apologize (33%) and transfer authority in advice contexts, with significant variation across model types and task domains.
arXiv:2606. 11635v1 Announce Type: cross Abstract: For highly capable AI systems to operate safely in dynamic, open-ended environments, they must be able to identify, understand, and respond to moral reasons for action, and constrain their behaviour accordingly.
arXiv:2601. 14295v4 Announce Type: replace Abstract: Large language models increasingly function as artificial reasoners: they evaluate arguments, assign credibility, and express confidence.
arXiv:2608. 12585v1 Announce Type: new Abstract: Improving reasoning LLMs requires the ability to judge the quality of long reasoning traces for effective reasoning data curation, strong training signals during reinforcement learning, and an in-depth understanding of reasoning behaviors during model performance evaluation.
The paper proposes a theory for judging post-hoc debates in AI, focusing on properties like reproducibility, robustness, groundedness, and explainability. It evaluates two debate‑judgement methods—LLM judges and formal computational argumentation semantics—finding similar accuracy but noting that argumentation semantics offers stronger formal guarantees. The study suggests that argumentation semantics is a preferable framework for principled debate judges in AI systems.