arXiv Statistics ML

Instance-Optimal Estimation with Multiple LLM Judges on a Budget

The paper addresses how to allocate a fixed evaluation budget across heterogeneous large language model judges to estimate scores for multiple prompt-response pairs most accurately. It introduces an inverse‑variance weighted estimator (IVWE) and an adaptive algorithm, EST‑IVWE, that uses optimistic variance estimates to approximate the oracle allocation. The authors prove that EST‑IVWE achieves near‑oracle error rates and establish a local minimax lower bound, demonstrating asymptotic instance‑optimality up to logarithmic factors, and validate the method on synthetic data and HelpSteer2 evaluations.

arXiv Machine Learning
Sep 23

Error Bounds for Statistical Estimators in BTL Model with Parametric Multivariate Utility Functions

The paper investigates preference elicitation under the Bradley‑Terry‑Luce model, focusing on estimating an unknown partworth vector from pairwise queries that satisfy a joint identifiability condition. It derives minimax lower bounds and shows that the canonical maximum likelihood estimator (MLE) exists, is unique, and achieves near‑optimal error rates once the sample size exceeds a design‑dependent threshold, without requiring compactness constraints or external regularizers. The analysis decomposes the estimation error into a linear stochastic term, a second‑order bias, and a higher‑order remainder, providing a unified non‑asymptotic theory for parametric utility elicitation.

By Yicheng Li, Huifu Xu
arXiv Machine Learning
Jul 7

Reliability and Identifiability in Persona-Trained Monte Carlo: Variance Decomposition, Stability Bounds, and the Identifiability of Heterogeneous News Reaction

arXiv:2607. 04627v1 Announce Type: new Abstract: Persona-Trained Monte Carlo (PTMC) estimates distributions of market-outcome functionals by repeatedly simulating limit-order-book interaction among $K$ neural policy bots whose behavioral personas are drawn from a learned heterogeneity distribution $\mathcal{P}$.

By Salavat Ishbulatov
arXiv Machine Learning
Jul 9

Best-Arm Identification with Generative Proxy

arXiv:2607. 06879v1 Announce Type: new Abstract: Best-arm identification is a canonical model for data-driven decision-making, but in many applications each reward observation is costly.

By Tianyi Ma, Hanzhang Qin, Ruihao Zhu, Jierui Zuo
arXiv Machine Learning
Oct 8

Efficient Best-of-N policy evaluation for inference-time alignment

The paper introduces a sample‑only framework for evaluating and selecting Best‑of‑N (BoN) inference‑time alignment policies without needing response likelihoods. By exploiting the order‑statistic structure of BoN, the authors express required density ratios through score‑rank probabilities that can be estimated from samples alone, and develop a doubly robust estimator (BoN‑DR) that reuses a shared auxiliary sample pool across candidate budgets. The method provides valid asymptotic inference even with reward estimator misspecification, proves efficiency, and offers two selection rules—maximizing estimated policy value or a lower confidence bound—to avoid over‑optimization and guarantee no harm, as demonstrated on synthetic data and GSM8K.

By Jonas Schweisthal, Yuxin Wang, Athiya Deviyani, Stefan Feuerriegel, Dennis Frauen
arXiv Statistics ML
Nov 9

Reweighting for Transfer Classification: Exact Risk Conversion and Minimax Adaptation

The paper investigates how reweighting a source training mixture affects target classification risk. It derives exact excess hinge‑risk bounds for binary classification, provides a two‑sample oracle inequality for weighted SVMs, and establishes minimax rates that separate source sampling uncertainty from relation uncertainty. The results show that the usefulness of a source depends on the training problem it creates, the relevance of its accuracy to the target, and the ability to apply corrective weighting.

By Aoran Chen, Yang Feng
arXiv Machine Learning
Nov 2

A Unified Framework for Probability Estimation of Data-Dependent Events

The paper introduces a unified framework for estimating the probability that an unknown distribution assigns to a data‑dependent event defined by a sample and a known mapping. It analyzes two classical estimators—the empirical plug‑in estimator and the leave‑one‑out (Good–Turing) estimator—showing how their errors relate to generalization gaps and stability under sample deletion. The framework is applied to error estimation for predictors that combine a base classifier with memorized exceptions, leading to new bounds based on VC dimension and mutual information for adaptive missing mass estimation.

By Vitaly Feldman, Satyen Kale, Vinod Raman, Kunal Talwar, Ambuj Tewari