LLM-based social simulation is a promising complement to traditional methods such as surveys and behavioral experiments. A core question is how to evaluate the fidelity of LLM-simulated human behavior...
The paper introduces NAPHA, a lightweight post‑hoc alignment method that improves large language model (LLM) predictions of human judgment distributions (HJD) by matching LLM output distributions to HJD through entropy‑based class assignment and specialized alignment models. Experiments on five datasets show that while LLMs perform near human‑level on hard‑label tasks, they struggle with soft‑label predictions, and NAPHA consistently enhances soft‑label accuracy, especially on high‑entropy instances. The study also demonstrates that better entropy class prediction can further boost NAPHA’s effectiveness.
By Sebastian Steindl, Nikos Voskarides, Alberto Gasparin, Diego Marcheggiani
The paper investigates the mismatch between large language models’ verbalized (knowledge) distributions and their actual output (behavior) distributions when predicting group opinions. It shows that for 37% of World Values Survey questions, the most likely answer under the verbalized distribution differs from that under the log‑probability distribution, indicating a disconnect between what the model knows and what it outputs. To reconcile this, the authors propose a learnable combination of the two distributions that preserves the knowledge shape while aligning with the behavior’s most likely answer, achieving higher behavioral consistency with only a modest increase in distance to human opinion distributions.
By Yang Liu, Masahiro Kaneko, Chenhui Chu
arXiv:2606. 14199v1 Announce Type: cross Abstract: Large language models are increasingly deployed as human simulators for interactive evaluation and social simulation.
By Xuhui Zhou, Weiwei Sun, Weihua Du, Jiarui Liu, Haojia Sun, Qianou Ma, Tongshuang Wu, Yiming Yang, Maarten Sap
The paper investigates whether fine‑tuning large language models (LLMs) on value survey responses can shift their value‑conditioned outputs and downstream behavior. Researchers first estimate baseline value profiles for several open‑source LLMs using the ten Schwartz values, then fine‑tune the models on survey‑style supervision that selectively alters target value ratings while preserving non‑target values. The resulting shifts are evaluated on held‑out survey questions and on out‑of‑domain behavioral proxies, such as a contextual moral‑judgment benchmark from Reddit AITA posts and a text‑based interactive decision‑making benchmark. The study finds that survey‑based supervision can change both survey responses and downstream behavior, but the transfer is strongly model‑ and value‑dependent and influenced by how value descriptions are framed.
By Shangrui Nie, Florian Mai, David Kacz\'er, Charles Welch, Zhixue Zhao, Lucie Flek
arXiv:2511. 14117v2 Announce Type: replace Abstract: Supervised classifiers output a distribution over classes but are typically trained against a single label obtained by collapsing multiple annotators into a majority vote.
By Agamdeep Singh, Ashish Tiwari, Hosein Hasanbeig, Priyanshu Gupta