arXiv Machine Learning

Democratic ICAI: Debating Our Way to Steering Principles from Preferences

arXiv:2606. 28294v1 Announce Type: new Abstract: Preference-based alignment often struggles to capture the reasoning that underlies human judgments.

arXiv AI
Sep 4

Toward Collective-Centric Evaluation of Preference Inference for Participatory Democracy

The paper examines how Preference Inference (PI) models used in large-scale participatory democracy platforms can alter the perceived consensus and minority support by predicting missing votes. It introduces a collective‑centric evaluation framework that assesses whether inferred votes maintain key properties of the overall preference landscape, rather than focusing solely on individual prediction accuracy. Using the largest multilingual dataset to date—four consultations with over 90,000 participants, 1 million votes, and 22 languages—the study finds that models with similar predictive accuracy can differ markedly in how well they preserve the collective structure, underscoring that accuracy alone is insufficient for evaluating PI in democratic contexts.

By Pierre-Antoine Lequeu, Salim Hafid, Paul Lerner, Nazanin Shafiabadi, Laur\`ene Cave, David Mas, Jean-Philippe Cointet, Benjamin Piwowarski, Fran\c{c}ois Yvon
arXiv AI
Sep 18

AI Should Facilitate Democratic Deliberation at Scale

The paper argues that AI can strengthen democracy by supporting large‑scale deliberation, addressing cognitive, social, platform‑design, and market frictions while preserving human agency. It contrasts AI‑assisted deliberation with liquid democracy, claiming the former lowers barriers to meaningful engagement without replacing human judgment. The authors outline four guiding principles—preserving agency, encouraging mutual respect, promoting equality, and augmenting active citizenship—and discuss challenges such as alignment, sycophancy, bias, and over‑reliance. They call on the machine learning community to develop and evaluate deliberation‑focused AI systems based on their ability to facilitate informed, representative, and friction‑robust discourse.

By Jos\'e Ram\'on Enr\'iquez, Jiaxin Pei, Alex Pentland
arXiv Computation and Language
Sep 1

Evaluating the Capabilities of LLMs for Persuasive Dialogue

The paper introduces “Persuasio”, a multi‑agent dialogue platform that uses a formal argumentation theory to adjudicate winners in free‑text debates. Using this system, the authors generated 192 debates on a UK political topic involving humans and large language models (LLMs), and evaluated 22 interlocutors through automated adjudication and 9,702 crowdsourced pairwise judgments across 1,386 annotation instances. The results show a consistent decoupling between subjective persuasiveness—where LLMs dominate—and formal argumentative strength—where humans remain competitive, with multi‑agent and retrieval‑augmented variants widening this gap.

By Jordan Robinson, Angus R. Williams, Katie Atkinson, Anthony G. Cohn
arXiv AI
4d ago

PADM\'E: Preference Alignment Data Synthesis for Meta-Evaluation of LM Agent Evaluators

PADM'E is a method for synthesizing preference‑aligned data to meta‑evaluate language‑model (LM) evaluators of agentic behaviors. It reframes meta‑evaluation as a preference judgment problem, generating criterion‑based data with small LMs and no human involvement. In a prototype, PADM'E produced 1,000 samples across four domains and three criteria, and human validation showed agreement with human judgment rising from 73% to 85% compared to a naive baseline.

By Cheng Chang, Yining Mao, Peng Qi