arXiv:2607. 26348v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as synthetic users, stand-ins for human respondents whose simulated answers feed product, policy, and market decisions.
The study evaluates whether large language models (LLMs) used as synthetic personas can predict real audience responses to marketing copy. Using thousands of headline A/B tests from the Upworthy Research Archive, the authors compare a ten-persona panel grounded in real audience demographics to a no-persona zero‑shot baseline that asks the model for a typical reader’s click likelihood. Results show that the no‑persona baseline outperforms the persona‑based approach, with higher predictive validity and top‑1 accuracy, indicating that forcing the model to role‑play specific personas introduces bias and noise.
Researchers use synthetic survey respondents generated by large language models as substitutes for human samples, but current validation methods often compare them to human surveys in ways that may not reflect real-world consequential behaviour. The authors propose a new validation framework that requires explicit statements of how well synthetic data correspond to human behaviour, specifies which diagnostics are addressed, and demands subgroup-level validity claims to avoid misrepresentation. The framework operationalises distributional, procedural, and recognition justice dimensions and introduces within-persona counterfactual experiments, illustrated with a case study on electric vehicle charging tariffs and concluded with a reporting checklist for researchers.
The paper discusses the use of synthetic data in marketing research, arguing that the key question is not whether synthetic respondents work, but when they do. It categorizes synthetic data into three types—ungrounded LLM responses, segment-level personas, and individual-level digital twins—and maps each to the decisions they can support. The authors also propose a taxonomy of accuracy measures, highlight the forgotten question problem, and introduce an ex‑ante answerability diagnostic based on R² to improve twin-human correlation.
arXiv:2609.15849v1 Announce Type: cross Abstract: Can LLMs reason through new information like humans, or do they merely retrieve cached opinions? This is critical for silicon sampling, where LLM per...
arXiv:2607. 11269v1 Announce Type: new Abstract: Decision support systems (DSS) increasingly run retention what-if analysis on synthetic customer populations, because privacy constraints preclude unrestricted use of real data.
arXiv:2607. 20787v1 Announce Type: cross Abstract: For two decades, the standard remedy for class-imbalanced learning has been to fabricate synthetic minority examples, and the standard evidence of their validity has been a check that cannot fail: synthetic points are scored against the very data that generated them.
The study audits demographic bias across four deep knowledge tracing architectures—DKT, DKVMN, SAKT, and AKT—using two large public datasets (Eedi and OULAD). It finds that bias is context‑dependent: socioeconomic bias is significant on Eedi, while gender bias appears on OULAD for most models. The most accurate model, AKT, also exhibits the greatest bias, and standard mitigation techniques such as reweighting and adversarial debiasing fail to reduce bias without sacrificing accuracy.
arXiv:2610.35698v1 Announce Type: new Abstract: For large language models (LLMs) and multimodal large language models (MLLMs), model-reported self-confidence is often miscalibrated, yet it may still...
Can Humans Tell? A Dual-Axis Study of Human Perception of LLM-Generated News investigates whether people can distinguish news articles written by humans from those generated by large language models (LLMs). Using JudgeGPT, the study collected 2,318 judgments from 1,054 participants on content from six LLMs, finding that participants could not reliably differentiate machine-generated from human-written text. The research also shows that domain expertise predicts judgment accuracy, distinct response strategies exist, and accuracy declines after about 30 evaluations due to cognitive fatigue.
The study evaluates a Korean synthetic persona panel (NVIDIA Nemotron‑Personas‑Korea) conditioned on Gemini 3.5 Flash and EXAONE against the KISDI Korea Media Panel Survey. Across eight digital‑AI service‑use indicators and eight innovativeness constructs, the panels achieved mean absolute errors of 15–19 pp, with segment‑level errors up to 52.4 pp and correlation coefficients between 0.69 and 0.90. Holdout calibration reduced sex‑by‑age cell errors but still lagged behind direct real‑data estimation, indicating that synthetic panels are useful diagnostically but not as survey substitutes.
arXiv:2604. 22891v4 Announce Type: replace-cross Abstract: LLM-as-a-Judge has become a dominant approach in automated evaluation systems, playing critical roles in model alignment, leaderboard construction, quality control, and so on.