arXiv:2605. 02122v2 Announce Type: replace-cross Abstract: Human evaluation remains the primary standard for assessing modern AI systems, yet annotator disagreement, bias, and variability make system rankings fragile under standard majority vote aggregation.
The paper titled "Whose Ground Truth? Embracing Ambiguity in Human-Centered AI" argues that current AI systems assume a single definitive ground truth, which ignores the inherent ambiguity in human interpretations. It calls for modeling the full space of plausible human judgments, distinguishing true ambiguity from noise, and suggests that this shift should influence how AI is represented, learned, evaluated, deployed, and governed.
arXiv:2608. 10327v1 Announce Type: new Abstract: Can AI systems be aligned to human values?
arXiv:2607. 14240v1 Announce Type: new Abstract: Current alignment approaches typically focus on emulating human behavior using static representations of human preferences, failing to capture the dynamic, context-dependent nature of real-world human-AI interactions.
arXiv:2608. 03910v1 Announce Type: new Abstract: As AI systems are deployed across increasingly diverse social contexts, alignment can no longer be framed as the optimization of a single, unified set of values.
arXiv:2606. 12754v1 Announce Type: cross Abstract: Are large language models (LLMs) bad at capturing human judgment?
The paper examines whether AI‑driven debate can help humans and weaker language models (LLMs) correct their judgments. Experiments show that debate improves human accuracy over prior beliefs, especially with weighted voting, but offers little benefit—and sometimes harms—accuracy for weaker LLM supervisors. The study highlights debate as a promising tool for scalable human oversight while questioning its effectiveness for fully automated AI control.
We are improving our AI systems’ ability to learn from human feedback and to assist humans at evaluating AI. Our goal is to build a sufficiently aligned AI system that can help us solve all other alignment problems.
OpenAI surveyed over 1,000 people worldwide on how AI should behave and compared their views to our Model Spec. Learn how collective alignment is shaping AI defaults to better reflect diverse human values and perspectives.
The paper discusses how human feedback used to align language models aggregates conflicting judgments about desirable behavior, resulting in an objective that reflects a collective preference. It surveys recent research that applies social choice theory to this aggregation problem, showing how this perspective can uncover failure modes in the feedback layer and expand the design space for managing disagreement in a principled manner.
The paper outlines a roadmap for advancing pluralistic value alignment in AI, noting that current research lacks evidence of influence on model training or evaluation. It identifies three key barriers: a need for empirical studies on benefits and drawbacks, clearer specifications of when and how pluralistic behavior should occur, and robust evaluation methods suitable for production. The roadmap aims to guide research toward practical deployment in responsible AI systems.
arXiv:2608. 12372v1 Announce Type: new Abstract: AI systems are increasingly employed as decision aids, decision delegates, or autonomous decision-makers.