arXiv Statistics ML By Junghyun Lee, Sanghwa Kim, Yassir Jedra, Alexandre Prouti\`ere, Se-Young Yun

Instance-Optimal Estimation with Multiple LLM Judges on a Budget

Read the original on arXiv Statistics ML →

The paper addresses how to allocate a fixed evaluation budget across heterogeneous large language model judges to estimate scores for multiple prompt-response pairs most accurately. It introduces an inverse‑variance weighted estimator (IVWE) and an adaptive algorithm, EST‑IVWE, that uses optimistic variance estimates to approximate the oracle allocation. The authors prove that EST‑IVWE achieves near‑oracle error rates and establish a local minimax lower bound, demonstrating asymptotic instance‑optimality up to logarithmic factors, and validate the method on synthetic data and HelpSteer2 evaluations.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Statistics ML.

arXiv Machine Learning
Sep 23

Error Bounds for Statistical Estimators in BTL Model with Parametric Multivariate Utility Functions

The paper investigates preference elicitation under the Bradley‑Terry‑Luce model, focusing on estimating an unknown partworth vector from pairwise queries that satisfy a joint identifiability condition. It derives minimax lower bounds and shows that the canonical maximum likelihood estimator (MLE) exists, is unique, and achieves near‑optimal error rates once the sample size exceeds a design‑dependent threshold, without requiring compactness constraints or external regularizers. The analysis decomposes the estimation error into a linear stochastic term, a second‑order bias, and a higher‑order remainder, providing a unified non‑asymptotic theory for parametric utility elicitation.

By Yicheng Li, Huifu Xu
arXiv Machine Learning
Jul 7

Reliability and Identifiability in Persona-Trained Monte Carlo: Variance Decomposition, Stability Bounds, and the Identifiability of Heterogeneous News Reaction

arXiv:2607. 04627v1 Announce Type: new Abstract: Persona-Trained Monte Carlo (PTMC) estimates distributions of market-outcome functionals by repeatedly simulating limit-order-book interaction among $K$ neural policy bots whose behavioral personas are drawn from a learned heterogeneity distribution $\mathcal{P}$.

By Salavat Ishbulatov
arXiv Machine Learning
Jul 9

Best-Arm Identification with Generative Proxy

arXiv:2607. 06879v1 Announce Type: new Abstract: Best-arm identification is a canonical model for data-driven decision-making, but in many applications each reward observation is costly.

By Tianyi Ma, Hanzhang Qin, Ruihao Zhu, Jierui Zuo
arXiv Machine Learning
Oct 8

Efficient Best-of-N policy evaluation for inference-time alignment

The paper introduces a sample‑only framework for evaluating and selecting Best‑of‑N (BoN) inference‑time alignment policies without needing response likelihoods. By exploiting the order‑statistic structure of BoN, the authors express required density ratios through score‑rank probabilities that can be estimated from samples alone, and develop a doubly robust estimator (BoN‑DR) that reuses a shared auxiliary sample pool across candidate budgets. The method provides valid asymptotic inference even with reward estimator misspecification, proves efficiency, and offers two selection rules—maximizing estimated policy value or a lower confidence bound—to avoid over‑optimization and guarantee no harm, as demonstrated on synthetic data and GSM8K.

By Jonas Schweisthal, Yuxin Wang, Athiya Deviyani, Stefan Feuerriegel, Dennis Frauen