arXiv AI
Jun 4
Generative Augmented Inference
arXiv:2604. 14575v2 Announce Type: replace-cross Abstract: Large language models enable inexpensive AI-generated annotations, but using them reliably for causal inference remains challenging.
Generative Augmented Inference of LLM-generated Data for Market Research: Theory and Empirical Evidence
arXiv:2604. 14575v3 Announce Type: replace-cross Abstract: Marketing research often relies on parameters estimated from costly human-generated data, such as conjoint survey responses, purchase decisions, and field experiment outcomes.
Related stories
arXiv AI
Aug 20
Debiased Inference for AI-Generated Data without Gold-Standard Labels: Identification via Multiple Imperfect Measurements
The paper introduces Debiased Inference with Multiple Imperfect Measurements (DMM), a framework that uses several error‑prone AI measurements to perform valid downstream statistical inference without requiring costly gold‑standard labels. By assuming conditional independence of the measurements given the true label and unit‑level features, DMM leverages CP decomposition and semiparametric theory to prove consistency and asymptotic normality of its estimator. Simulations demonstrate that DMM yields valid inference and can improve efficiency when additional imperfect measurements are available, and the authors provide diagnostics for the key independence assumption.
arXiv Machine Learning
Jul 21
BACON: Budgeted Human Calibration for Modeling and Evaluation with Multiple AI Judges
arXiv:2607. 16239v1 Announce Type: new Abstract: AI judges offer a scalable, low-cost alternative to human evaluation, but their outputs can be biased relative to human preferences and highly item-dependent, varying across judges, tasks, and domains.
arXiv AI
Jul 13
Rectification Difficulty and Optimal Sample Allocation in LLM-Augmented Surveys
arXiv:2604. 17267v2 Announce Type: replace Abstract: Large Language Models can generate synthetic survey responses at low cost, but their accuracy varies unpredictably across questions.
arXiv Machine Learning
Sep 1
AI-Generated Measurements for Identification and Inference with Missing Data: A Weak Shadow Variable Approach
The paper introduces an assumption‑lean framework that uses AI‑generated measurements as weak shadow variables to identify and infer population quantities when data are missing not at random. Weak shadow variables are outcome‑informative proxies that are conditionally independent of missingness given the true outcome and covariates, and they do not need to predict missing outcomes accurately. The authors derive sharp bounds via linear programs and propose a localized penalized estimator with a subsampling algorithm for confidence intervals, demonstrating in semi‑synthetic experiments that the resulting intervals are substantially narrower and more accurate than classical MNAR methods.
arXiv Machine Learning
Aug 11
Demystifying Prediction Powered Inference
arXiv:2601. 20819v2 Announce Type: replace-cross Abstract: Machine learning predictions are increasingly used to supplement incomplete or costly-to-measure outcomes in fields such as biomedical research, environmental science, and social science.
arXiv Computation and Language
Aug 25
What Language Models Know But Don't Say: Non-Generative Prior Extraction for Generalization
arXiv:2601.17609v3 Announce Type: replace Abstract: In domains like medicine and finance, large-scale labeled data is costly and often unavailable, leading to models trained on small datasets that st...
arXiv Machine Learning
Sep 14
Aligning Language Models with Observational Data: Opportunities and Risks from a Causal Perspective
The paper discusses how large language models (LLMs) can be fine‑tuned with observational data to improve alignment with human preferences and business goals. It highlights that directly using such data can cause models to learn spurious correlations, and introduces DeconfoundLM, a method that removes known confounders from reward signals. Experiments show that DeconfoundLM better recovers causal relationships and outperforms baseline methods by over 16% in objective score when confounding is present.
arXiv Machine Learning
Jun 26
No Free Lunch: Non-Asymptotic Analysis of Prediction-Powered Inference
arXiv:2505. 20178v2 Announce Type: replace-cross Abstract: Prediction-Powered Inference (PPI) is a popular strategy for combining gold-standard and possibly noisy pseudo-labels to perform statistical estimation.
arXiv AI
Jun 30
HERO: Improving the Reliability and Sensitivity of Generative Model Evaluation Using Historical Data
arXiv:2606. 29784v1 Announce Type: cross Abstract: Reliable generative AI models critically rely on expert human annotations to evaluate output quality, yet these "gold" labels are expensive to collect and limited in quantity.
arXiv Machine Learning
Sep 25
A Probabilistic Approach for Model Alignment with Human Comparisons
The paper proposes a two‑stage framework, SL+LHF, that first learns low‑dimensional representations from noisy labeled data and then refines model alignment using human comparison feedback via a probabilistic bisection approach. It introduces the label‑noise‑to‑comparison‑accuracy (LNCA) ratio to theoretically identify when this framework outperforms pure supervised learning, showing that trading labels for comparisons reduces sample complexity when labels are scarce. Experiments on a high‑dimensional crowdfunding prediction task and an Amazon Mechanical Turk study confirm that incorporating human or large language model evaluators improves accuracy under a fixed query budget.
arXiv Machine Learning
Jun 26
Learning from a Biased Sample
arXiv:2209. 01754v5 Announce Type: replace-cross Abstract: The empirical risk minimization approach to data-driven decision making requires access to training data drawn under the same conditions as those that will be faced when the decision rule is deployed.