arXiv AI

Synthetic Data in Marketing Research: How to Evaluate and When to Trust

The paper discusses the use of synthetic data in marketing research, arguing that the key question is not whether synthetic respondents work, but when they do. It categorizes synthetic data into three types—ungrounded LLM responses, segment-level personas, and individual-level digital twins—and maps each to the decisions they can support. The authors also propose a taxonomy of accuracy measures, highlight the forgotten question problem, and introduce an ex‑ante answerability diagnostic based on R² to improve twin-human correlation.

arXiv AI
Jun 4

Synthetic Personalities: How Well Can LLMs Mimic Individual Respondents Using Socio-Economic Microdata?

arXiv:2606. 04592v1 Announce Type: cross Abstract: LLM-based digital twins promise to scale and accelerate market research, but most published twins are either coarse persona bots conditioned on a few demographic questions or detailed individual-level twins built on purpose-collected surveys and interview transcripts.

By Leonard Kinzinger, Jochen Hartmann
arXiv AI
Sep 25

AI-Moderated Interviews for Market Research and Digital Twins Calibration

AI‑moderated interviews are a scalable market‑research method that can match human moderation in depth, cover more themes, and recover more customer needs while keeping budgets constant. Participants, however, feel more emotionally engaged with live humans. Digital twins built from AI‑moderated data predict consumer responses better than demographics‑only personas, but the added richness does not improve quantitative predictions over static interviews, and prediction errors stem from differences in thinking styles and data‑distribution gaps.

By Yuting Deng, Jingxuan Liu, Olivier Toubia, Naman Jain
arXiv Computation and Language
2d ago

SYN-DIGITS: A Synthetic Control Framework for Calibrated Digital Twin Simulation

SYN-DIGITS is a model‑agnostic, post‑hoc calibration framework that uses relationships learned from digital twin simulations to predict human responses, especially in cold‑start scenarios where no target question data is available. It transfers these relationships to improve individual‑level correlation and distributional predictions, even when only marginal reference data is available. Experiments on three datasets and thirteen persona constructions show significant gains—up to 417% improvement in correlation and 57% reduction in distributional error—over uncalibrated baselines.

By Grace Jiarui Fan, Chengpiao Huang, Tianyi Peng, Kaizheng Wang, Yuhang Wu
arXiv Computation and Language
Sep 23

Do Synthetic Personas Predict Real Audience Response? A Sim-to-Real Study Where a No-Persona Baseline Beats Persona-Based Copy Simulation

The study evaluates whether large language models (LLMs) used as synthetic personas can predict real audience responses to marketing copy. Using thousands of headline A/B tests from the Upworthy Research Archive, the authors compare a ten-persona panel grounded in real audience demographics to a no-persona zero‑shot baseline that asks the model for a typical reader’s click likelihood. Results show that the no‑persona baseline outperforms the persona‑based approach, with higher predictive validity and top‑1 accuracy, indicating that forcing the model to role‑play specific personas introduces bias and noise.

By Alexandre Cristov\~ao Maiorano
arXiv Computation and Language
Sep 24

Consequential Behaviour and Representational Fairness in the Validation of Synthetic Research

Researchers use synthetic survey respondents generated by large language models as substitutes for human samples, but current validation methods often compare them to human surveys in ways that may not reflect real-world consequential behaviour. The authors propose a new validation framework that requires explicit statements of how well synthetic data correspond to human behaviour, specifies which diagnostics are addressed, and demands subgroup-level validity claims to avoid misrepresentation. The framework operationalises distributional, procedural, and recognition justice dimensions and introduces within-persona counterfactual experiments, illustrated with a case study on electric vehicle charging tariffs and concluded with a reporting checklist for researchers.

By Florian Kutzner, Celina Kacperski, Laura de Moli\`ere, Edoardo Chidichimo, Min Jun Jung, Felix Patrick Sedgwick Wallis, James Kunling He
arXiv AI
Oct 2

What Does a Sharing Question Add? Auditing LLM Survey Scores for Misinformation

The study examines whether asking large language models (LLMs) to predict whether readers would share misinformation adds useful information beyond predicting credibility. Eight LLM versions were audited on 290 synthetic misinformation articles, yielding 1,256 paired survey responses. While credibility scores explained a substantial portion of sharing variation, adding sharing predictions to credibility did not meaningfully reduce error, indicating that sharing answers contain structured variation but lack established incremental validity for human sharing.

By Zonghuan Xu, Xiang Zheng, Yutao Wu, Xingjun Ma