The paper introduces Population Fidelity, an evaluation framework for assessing how well large language models (LLMs) represent human population attitudes. It focuses on three dimensions: group-level accuracy, between-group variation, and the structure of that variation. Using the framework, the authors replicate a prior study on machine bias and test cultural fine-tuning, finding that while fine-tuning improves overall alignment, it does not enhance representation of within-population differences.
By Neemias B. da Silva, Martin Lukk, Ali Sutani, Abhishek Moturu, Harris Yang, Daniel Silver, Matt Ratto, Thiago H. Silva
The paper introduces Demographic Pluralism, an inference-time framework that estimates population-level opinion distributions without requiring training data or task-specific fine-tuning. It generates multiple perspectives within demographically grounded groups and, across four model backbones on GlobalOpinionQA and VITAL, reduces Jensen-Shannon distance by 8.4%–26.4% compared to Modular Pluralism. The study finds that equal weighting of group perspectives yields the best overall performance, while weighted aggregation performs worse due to increased group-level error.
By Meng-Chen Wu, Qipin Chen, Ansh Jain, Tess Wood, Zhe Du, Si-Chi Chin
arXiv:2606. 10569v1 Announce Type: cross Abstract: Standard RLHF pipelines often reduce heterogeneous human judgments into a single scalar reward target.
By Dorcas Chia Ern Chua, Karen Myn Hui Lee, Jia Yue Tan, Zhen Xue Gue, Norzalena Abdul Hamid, Azima Binti Azmi, Keat Mei Yeong, Aizat Izyani binti Mujab, Hafsah Noor Azam, Chee Guo Khoo, Han Ying Lim, Chee Seng Chan
Large language models (LLMs) are used worldwide, yet disproportionately reflect Western values, limiting their ability to represent diverse value systems. We introduce PLURAL, a large-scale, value-focused preference dataset grounded in the Integrated Values Survey (IVS), a nationally representative survey spanning 92 countries.
arXiv:2607. 08034v1 Announce Type: cross Abstract: Large language models (LLMs) are used worldwide, yet disproportionately reflect Western values, limiting their ability to represent diverse value systems.
By Dhruv Agarwal, Anya Shukla, Tanya Goyal, Aditya Vashistha
arXiv:2609.00565v1 Announce Type: cross
Abstract: Cultural fine-tuning has become the de facto paradigm for building culture-aware large language models (LLMs), yet existing optimization exclusively...
By Jingshen Zhang, Shaoyang Xu, Wenxuan Zhang
The paper examines how large language models (LLMs) respond to different demographic cues—such as names—when users seek advice, focusing on race and gender in a U.S. context. It finds that using different cues for the same group leads to only partially overlapping changes in model responses, producing inconsistent conclusions about personalization and unstable bias metrics. The authors argue that LLMs react to linguistic signals tied to cues rather than to stable demographic categories, and they call for evaluations that use multiple cues and consider underlying mechanisms.
By Manuel Tonneau, Neil K. R. Sehgal, Niyati Malhotra, Sharif Kazemi, Victor Orozco-Olvera, Ana Mar\'ia Mu\~noz Boudet, Lakshmi Subramanian, Samuel P. Fraiberger, Sharath Chandra Guntuku, Valentin Hofmann
Group alignment adapts a language model to a demographic group to produce responses that reflect the group's opinions, values, and preferences. Sycophancy, a well-documented by-product of alignment, causes the model to over-agree with the user regardless of factual and objective information.
arXiv:2607.19231v2 Announce Type: replace
Abstract: Prior work on human label variation (HLV) in natural language inference (NLI) has often relied on re-annotation resources that select items by disa...
By Haram Choi
arXiv:2607. 26348v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as synthetic users, stand-ins for human respondents whose simulated answers feed product, policy, and market decisions.
By Zihan Chen, Di Zhu, Lei Nico Zheng
arXiv:2605.12515v3 Announce Type: replace
Abstract: Multilingual large language models (MLLMs) often answer culturally ambiguous questions inconsistently across languages, even for a fixed user perso...
By Lucas Resck, Isabelle Augenstein, Anna Korhonen
arXiv:2609.00222v1 Announce Type: new
Abstract: Large language models (LLMs) are increasingly used as judges for subjective tasks, where annotators disagree and the relevant question is not only how...
By Daniela Occhipinti, Andrea Piergentili, Marco Guerini