Silicon Samples Match the Answers but Not the Respondents
Read the original on arXiv Computation and Language →Silicon sampling creates synthetic survey respondents by having large language models answer questions as people with specified demographics, and these synthetic populations are increasingly replacing real respondents. Current evaluations focus on answer accuracy and distributional matches but do not assess whether the synthetic respondents resemble actual human respondents. The authors introduce the respondent coverage gap (RCG) metric, which measures how far each real respondent is from the nearest synthetic one, and find that all six evaluated LLMs leave human respondents uncovered, performing worse than a baseline that samples answers independently from human marginals.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.