The paper introduces “Persuasio”, a multi‑agent dialogue platform that uses a formal argumentation theory to adjudicate winners in free‑text debates. Using this system, the authors generated 192 debates on a UK political topic involving humans and large language models (LLMs), and evaluated 22 interlocutors through automated adjudication and 9,702 crowdsourced pairwise judgments across 1,386 annotation instances. The results show a consistent decoupling between subjective persuasiveness—where LLMs dominate—and formal argumentative strength—where humans remain competitive, with multi‑agent and retrieval‑augmented variants widening this gap.
By Jordan Robinson, Angus R. Williams, Katie Atkinson, Anthony G. Cohn
arXiv:2505. 22961v3 Announce Type: replace-cross Abstract: Large language models (LLMs) have shown promising potential in persuasion, but existing works on training LLM persuaders are still preliminary.
By Peixuan Han, Zijia Liu, Jiaxuan You
arXiv:2606. 05330v1 Announce Type: cross Abstract: Large language models can shift human beliefs across high-stakes domains, but most persuasion studies rely on pre/post belief change.
By Jared Moore, Noah Goodman, Nick Haber, Max Kleiman-Weiner
arXiv:2601. 14171v2 Announce Type: replace Abstract: Writing effective rebuttals is a high-stakes task that demands more than linguistic fluency, as it requires precise alignment between reviewer intent and manuscript details.
By Qianli Ma, Chang Guo, Zhiheng Tian, Siyu Wang, Jipeng Xiao, Yuanhao Yue, Zhipeng Zhang
The paper introduces the Persuasion Index (PI), a taxonomy of 15 persuasion dimensions grounded in psychological and communication theories, implemented with 55 lexicon- and rule-based sub-features. PI is modular, allowing individual detectors to be swapped while preserving its theoretical framework. Evaluations on four English argumentative datasets show that PI provides a shared, lightweight feature space that captures meaningful predictive signals and reveals consistent dimension-level associations with persuasion outcomes, with variations across topics and stances.
By Liancheng Gong, Zhiyang Wang, Yiwei Xu, Julia Mendelsohn
arXiv:2605.28313v2 Announce Type: replace
Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities in tasks related to reasoning and judgment. However, assessing the quality o...
By Nicol\'as Benjam\'in Ocampo, Agnes Paullate Nyiranziza, Davide Ceolin
arXiv:2606. 10296v1 Announce Type: cross Abstract: Multi-agent debate systems are typically evaluated only on whether the final answer is correct, overlooking the quality of the intermediate reasoning that debate is designed to produce.
By Ali Keramati, Justin Cheok, Jacob Horne, Mark Warschauer
Enthymemes, arguments with unstated premises or conclusions, are pervasive in persuasive discourse, yet their annotation remains notoriously subjective. We present a resource of 1,482 tweets from politically controversial discourse, annotated by five annotators for the presence of enthymemes and their argument structure, designed to study label variation.
arXiv:2606. 24976v1 Announce Type: cross Abstract: Foundation-model agents in multi-step, open-ended environments frequently suffer from compounding errors, where early mistakes contaminate long-horizon trajectories.
By Pradyumna Narayana, Sana Ayromlou, Purvi Sehgal
arXiv:2606. 10327v1 Announce Type: cross Abstract: Automated Essay Scoring (AES) systems must judge interdependent discourse elements (e.
By Ali Keramati, Mark Warschauer
Evaluating reasoning quality in multi-agent LLM systems is challenging, especially for open-ended tasks without reference answers. We investigate whether intrinsic confidence signals, token-level log-probabilities from decoding, can predict reasoning quality as assessed by LLM-as-judge evaluation.
Theory of Mind (ToM) benchmarks for Large Language Models (LLMs) typically rely on passive question-answering formats, but the deployment of LLMs in increasingly agentic and autonomous forms demands new evaluations. In this paper we evaluate an agent's ability to induce specific belief states in other agents by taking actions rather than using conversational persuasion, a capability we call Non-Conversational Planning ToM (NCP-ToM).