← Back to all news
arXiv AI By Daniel Halpern, Evi Micha, Ariel D. Procaccia, Benjamin Schiffer, Itai Shapira, Shirley Zhang

AI Alignment From Social Choice Perspectives

Read the original on arXiv AI →

The paper discusses how human feedback used to align language models aggregates conflicting judgments about desirable behavior, resulting in an objective that reflects a collective preference. It surveys recent research that applies social choice theory to this aggregation problem, showing how this perspective can uncover failure modes in the feedback layer and expand the design space for managing disagreement in a principled manner.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Computation and Language
Oct 26

Aligning Large Language Models with Human Feedback: Mathematical Foundations and Algorithm Design

arXiv:2610.31862v1 Announce Type: cross Abstract: This article provides an introduction to the mathematical foundations and algorithmic frameworks used to align Large Language Models (LLMs) with huma...

By Siliang Zeng, Luca Viano, Chenliang Li, Jiaxiang Li, Volkan Cevher, Markus Wulfmeier, Stefano Ermon, Alfredo Garcia, Mingyi Hong
llmsreinforcement-learningfine-tuningbenchmarkssafety
More like this →
arXiv AI
Aug 20

Preference Reasoning under Indeterminacy in Large Language Models

The paper titled "Preference Reasoning under Indeterminacy in Large Language Models" discusses how large language models (LLMs) are becoming decision‑making agents and the importance of reasoning over preferences for alignment, coordination, and collective intelligence. It highlights that real‑world preference reasoning is inherently indeterminate, with incomplete information and potentially no valid solutions. The authors formalize this challenge into epistemic indeterminacy (incomplete or expressive preferences) and structural indeterminacy (non‑existence of solutions under standard social choice concepts), and demonstrate that current LLMs fail to differentiate between determined and undetermined cases, showing miscalibrated reasoning even in verification tasks.

By Hadi Hosseini, Samarth Khanna, Xiyuan Wang
llmsagentsbenchmarkssafety
More like this →
arXiv AI
Jul 17

Align AI to Dynamic Human-AI Workflows

arXiv:2607. 14240v1 Announce Type: new Abstract: Current alignment approaches typically focus on emulating human behavior using static representations of human preferences, failing to capture the dynamic, context-dependent nature of real-world human-AI interactions.

By Valerie Chen, Cleotilde Gonzalez, Anita Williams Woolley, Michael Lee, Tongshuang Wu, Vincent Conitzer, Aarti Singh
safety
More like this →