arXiv Computation and Language
LLMs as survey respondents: Comparison to simple semantic baselines
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv Computation and Language
Sep 3
When Persona Attributes Improve Population Alignment in Large Language Models
The paper investigates how persona prompting—using short textual descriptions of individuals—to align large language models (LLMs) with human survey responses. It examines the impact of selecting different persona attributes and finds that not all attribute combinations improve performance, suggesting that the variation in human responses to survey questions may explain mixed results. The study evaluates multiple attribute selection methods across four social surveys, two countries, six LLMs, and twenty prediction tasks, offering guidance on when persona prompting is beneficial and which attribute choices are most effective.
arXiv AI
Jun 3
Whom to Query for What: Adaptive Group Elicitation via Multi-Turn LLM Interactions
arXiv:2602. 14279v2 Announce Type: replace-cross Abstract: Eliciting information to reduce uncertainty about latent group-level properties from surveys and other collective assessments requires allocating limited questioning effort under real costs and missing data.
arXiv AI
Oct 2
Certainty Is Not Just Correctness: Rethinking Token-Level Certainty in LLM Reasoning
The paper investigates token‑level certainty as a proxy for correctness in large language models. It finds that certainty better predicts whether a model will answer a question correctly than it does whether a specific response is correct, and that certainty varies by token type and position. The authors show that using certainty early in generation to allocate responses and later to weight votes improves accuracy while dramatically cutting token cost.
arXiv AI
Aug 25
Hybrid Panels: Toward Human-AI Collaboration in Survey Research
arXiv:2608.22582v1 Announce Type: cross Abstract: Large-scale population surveys are essential for generating robust social and scientific insights, yet they face significant challenges, including de...
arXiv Computation and Language
Oct 15
When Can Digital Personas Reliably Approximate Human Survey Findings?
arXiv:2605.10659v3 Announce Type: replace Abstract: Digital personas powered by Large Language Models (LLMs) are increasingly proposed as substitutes for human survey respondents, yet it remains uncl...
arXiv Computation and Language
3d ago
Speak vs. Know: Balancing Behavior and Knowledge Distributions of Large Language Models
The paper investigates the mismatch between large language models’ verbalized (knowledge) distributions and their actual output (behavior) distributions when predicting group opinions. It shows that for 37% of World Values Survey questions, the most likely answer under the verbalized distribution differs from that under the log‑probability distribution, indicating a disconnect between what the model knows and what it outputs. To reconcile this, the authors propose a learnable combination of the two distributions that preserves the knowledge shape while aligning with the behavior’s most likely answer, achieving higher behavioral consistency with only a modest increase in distance to human opinion distributions.