arXiv Machine Learning
When Can You Trust Your Synthetic Users? Diagnostics and Corrections for LLM Consumer Panels
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Jul 31
When Synthetic Users Fail: A Cross-Domain Benchmark of LLM-Simulated Human Survey Responses
arXiv:2607. 26348v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as synthetic users, stand-ins for human respondents whose simulated answers feed product, policy, and market decisions.
arXiv AI
4d ago
Synthetic Data in Marketing Research: How to Evaluate and When to Trust
The paper discusses the use of synthetic data in marketing research, arguing that the key question is not whether synthetic respondents work, but when they do. It categorizes synthetic data into three types—ungrounded LLM responses, segment-level personas, and individual-level digital twins—and maps each to the decisions they can support. The authors also propose a taxonomy of accuracy measures, highlight the forgotten question problem, and introduce an ex‑ante answerability diagnostic based on R² to improve twin-human correlation.
arXiv AI
4d ago
Before You Poll with LLMs: A Deliberative Diagnostic Framework
arXiv:2609.15849v1 Announce Type: cross Abstract: Can LLMs reason through new information like humans, or do they merely retrieve cached opinions? This is critical for silicon sampling, where LLM per...
arXiv Machine Learning
Jul 14
Trustworthy synthetic data for campaign decision support: strategy simulation fidelity and the PolicySynth framework
arXiv:2607. 11269v1 Announce Type: new Abstract: Decision support systems (DSS) increasingly run retention what-if analysis on synthetic customer populations, because privacy constraints preclude unrestricted use of real data.
arXiv AI
Jul 24
Synthetic minority data is redundant or invalid: a data-dependent validity theory and a de-biased test
arXiv:2607. 20787v1 Announce Type: cross Abstract: For two decades, the standard remedy for class-imbalanced learning has been to fabricate synthetic minority examples, and the standard evidence of their validity has been a check that cannot fail: synthetic points are scored against the very data that generated them.
arXiv Machine Learning
1d ago
Accuracy Is Not Enough: A Cross-Architecture Audit of Demographic Bias in Deep Knowledge Tracing
The study audits demographic bias across four deep knowledge tracing architectures—DKT, DKVMN, SAKT, and AKT—using two large public datasets (Eedi and OULAD). It finds that bias is context‑dependent: socioeconomic bias is significant on Eedi, while gender bias appears on OULAD for most models. The most accurate model, AKT, also exhibits the greatest bias, and standard mitigation techniques such as reweighting and adversarial debiasing fail to reduce bias without sacrificing accuracy.