arXiv:2609.24574v1 Announce Type: new
Abstract: Computational social science increasingly relies on large language models for text annotation, and the validity of published findings now rests on the...
By Hazem Ibrahim, Yasir Zaki
arXiv:2606. 27997v1 Announce Type: new Abstract: Benchmarks of machine learning models often include many datasets, making evaluation expensive.
By Rostislav Gusev, Alexey Zaytsev
arXiv:2608. 19323v1 Announce Type: cross Abstract: Uncertainty quantification (UQ) is essential for the safe deployment of large language models (LLMs).
By Sokhna Diarra Mbacke, Mouloud Belbahri, Gabriel Loaiza-Ganem
The paper introduces Debiased Inference with Multiple Imperfect Measurements (DMM), a framework that uses several error‑prone AI measurements to perform valid downstream statistical inference without requiring costly gold‑standard labels. By assuming conditional independence of the measurements given the true label and unit‑level features, DMM leverages CP decomposition and semiparametric theory to prove consistency and asymptotic normality of its estimator. Simulations demonstrate that DMM yields valid inference and can improve efficiency when additional imperfect measurements are available, and the authors provide diagnostics for the key independence assumption.
By Naoki Egami, Sooahn Shin
arXiv:2608.23086v1 Announce Type: new
Abstract: Black-box large language models need confidence scores that can separate likely-correct from likely-incorrect outputs, enabling systems to prioritize h...
By Rounak Sharma, Ananya B. Sai, Soumyabrata Pal
arXiv:2607. 19367v1 Announce Type: new Abstract: Calibration is the primary criterion for evaluating LLM confidence, but it is insufficient: it admits trivially incoherent estimators, depends on the evaluation distribution, and does not test the extent to which the estimation can be interpreted as a consistent, underlying probability function.
By Krish Matta, Atharv Naphade, Andy Zou