arXiv Machine Learning
Item-Mean Surrogates: Why Richer Persona Data Fail to Improve LLMs as Human Surrogates
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Computation and Language
2d ago
Item-Mean Surrogates: Why Richer Personas Fail to Deliver Individual Fidelity in LLMs as Human Surrogates
The study investigates whether large language models (LLMs) can accurately simulate individual human responses by testing the assumption that richer personas improve prediction fidelity. Across four public datasets, researchers found that LLMs largely predict item means but fail to capture individual deviations, particularly the stable, person‑by‑item signal that accounts for most variance. Even with fine‑tuning and varied models, LLMs recover only a small fraction of this nuanced signal, highlighting limitations in using them as precise human surrogates.
arXiv AI
Oct 15
Behavioral History Outperforms Descriptions of the Person for LLM Synthetic Personas
arXiv:2610.03998v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly used as synthetic personas representing survey respondents. Their validity as substitutes for particu...
arXiv AI
Jun 17
Statistical Foundations of LLM-based A/B Testing: A Surrogacy Framework for Human Causal Inference
arXiv:2606. 17165v1 Announce Type: cross Abstract: Organizations and researchers show increasing interest in using large language models (LLMs) in place of human participants in A/B tests, in the hope of experimenting faster and at lower cost.
arXiv Computation and Language
Oct 15
When Can Digital Personas Reliably Approximate Human Survey Findings?
arXiv:2605.10659v3 Announce Type: replace Abstract: Digital personas powered by Large Language Models (LLMs) are increasingly proposed as substitutes for human survey respondents, yet it remains uncl...
arXiv Computation and Language
Sep 23
Do Synthetic Personas Predict Real Audience Response? A Sim-to-Real Study Where a No-Persona Baseline Beats Persona-Based Copy Simulation
The study evaluates whether large language models (LLMs) used as synthetic personas can predict real audience responses to marketing copy. Using thousands of headline A/B tests from the Upworthy Research Archive, the authors compare a ten-persona panel grounded in real audience demographics to a no-persona zero‑shot baseline that asks the model for a typical reader’s click likelihood. Results show that the no‑persona baseline outperforms the persona‑based approach, with higher predictive validity and top‑1 accuracy, indicating that forcing the model to role‑play specific personas introduces bias and noise.
arXiv AI
Jul 8
An LLM-Native Psychometric Instrument Reveals a Self-Report--Behavior Gap Across 25 Models
arXiv:2606. 09843v3 Announce Type: replace-cross Abstract: Large language models (LLMs) give stable answers to personality questionnaires, yet these self-reports fail to predict how the models behave.