The paper introduces Pro-Active Inference (PAI), a method for efficiently estimating a population mean when only a subset of outcomes can be observed. PAI proactively selects which observation to label next from a covariate pool, updating machine‑learning predictions online, and provides finite‑sample unbiasedness, asymptotic validity, and an improved variance estimator. Experiments on eight datasets show that PAI outperforms Sequential Active Inference in most settings, while diagnostics reveal when additional adaptivity is beneficial or detrimental.
arXiv:2601. 20819v2 Announce Type: replace-cross Abstract: Machine learning predictions are increasingly used to supplement incomplete or costly-to-measure outcomes in fields such as biomedical research, environmental science, and social science.
arXiv:2607. 08347v1 Announce Type: cross Abstract: Active testing provides a label--efficient approach to risk estimation by adaptively selecting which test points should be labelled.
arXiv:2505. 20178v2 Announce Type: replace-cross Abstract: Prediction-Powered Inference (PPI) is a popular strategy for combining gold-standard and possibly noisy pseudo-labels to perform statistical estimation.
Active testing provides a label--efficient approach to risk estimation by adaptively selecting which test points should be labelled. However, existing estimators fail to exploit the informative predictions of powerful black--box models, even though such predictions are increasingly available in settings where labels remain expensive.
The paper introduces Conditional Poisson Sampling (CPS) as a fixed-size alternative to Poisson Sampling (PS) for variance reduction in statistical inference. CPS can replace PS in multivariate Z-estimation problems, including convex M-estimation, without increasing asymptotic variance and often improving it when predictions are imperfect. The authors demonstrate these benefits on synthetic and real datasets and provide a Python package for CPS implementation.
arXiv:2603.05575v2 Announce Type: replace-cross Abstract: We study prediction-powered conditional inference in the setting where labeled data are scarce, unlabeled covariates are abundant, and a blac...
arXiv:2609.38902v1 Announce Type: cross Abstract: Augmenting small concurrent studies with external or historical cohorts is attractive in drug development, where enrollment is slow, follow-up is exp...
arXiv:2609.35815v1 Announce Type: cross Abstract: Researchers across academia increasingly base significance claims on LLM judge scores and small-sample AI evaluations. Yet without well-calibrated co...
The paper introduces Debiased Inference with Multiple Imperfect Measurements (DMM), a framework that uses several error‑prone AI measurements to perform valid downstream statistical inference without requiring costly gold‑standard labels. By assuming conditional independence of the measurements given the true label and unit‑level features, DMM leverages CP decomposition and semiparametric theory to prove consistency and asymptotic normality of its estimator. Simulations demonstrate that DMM yields valid inference and can improve efficiency when additional imperfect measurements are available, and the authors provide diagnostics for the key independence assumption.
INCONVENIENT is a new benchmark that evaluates how machine‑learning models, especially large language models, generalize when trained on convenience samples that bias feature–target relationships. Unlike prior distribution‑shift tests that rely on natural domain splits, INCONVENIENT explicitly perturbs these relationships while keeping the deployment population unchanged, allowing controlled study of shift severity and direction. The benchmark covers 200 tabular classification and regression datasets and produces about 15,500 shifted training distributions, revealing significant performance degradation.
arXiv:2609.37687v1 Announce Type: new Abstract: Active test-time adaptation (ATTA) improves robustness under distribution shift by updating a deployed model during inference while selectively queryin...
arXiv:2602. 15327v2 Announce Type: replace-cross Abstract: Machine learning model performance improvements tend to arise from competition and application.