arXiv Machine Learning By Vitaly Feldman, Satyen Kale, Vinod Raman, Kunal Talwar, Ambuj Tewari

A Unified Framework for Probability Estimation of Data-Dependent Events

Read the original on arXiv Machine Learning →

The paper introduces a unified framework for estimating the probability that an unknown distribution assigns to a data‑dependent event defined by a sample and a known mapping. It analyzes two classical estimators—the empirical plug‑in estimator and the leave‑one‑out (Good–Turing) estimator—showing how their errors relate to generalization gaps and stability under sample deletion. The framework is applied to error estimation for predictors that combine a base classifier with memorized exceptions, leading to new bounds based on VC dimension and mutual information for adaptive missing mass estimation.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning
Oct 7

Assumption-lean logistic regression with missing covariates

arXiv:2610.07292v1 Announce Type: cross Abstract: Missing covariates are frequently encountered in supervised learning problems, and classical methods for estimation using such data use carefully cho...

By Jyotishka Ray Choudhury, Kabir Aladin Verchand, Richard J. Samworth, Ashwin Pananjady
arXiv Machine Learning
Oct 22

Approximate Solomonoff Induction: On the Robustness of Learned Universal Predictors

The paper investigates how well practical machine‑learning algorithms approximate Solomonoff’s universal predictor when trained by minimizing log‑loss over diverse datasets. It establishes tight bounds on the worst‑case increase in prediction error caused by approximating the Bayesian mixture predictor, showing that while the excess error can be large in low‑prior environments, it remains bounded on average and under distribution shifts. The authors also extend their analysis to finite‑sample settings relevant to neural networks and Transformers.

By Qian Li, Shang-Hua Teng
arXiv Machine Learning
Jun 9

Partial Identification under Missing Data Using Weak Shadow Variables from Pretrained Models

arXiv:2602. 16061v2 Announce Type: replace-cross Abstract: Estimating population quantities such as mean outcomes from user feedback is fundamental to platform evaluation and social science, yet feedback is often missing not at random (MNAR): users with stronger opinions are more likely to respond, so standard estimators are biased and the estimand is not identified without additional assumptions.

By Hongyu Chen, David Simchi-Levi, Ruoxuan Xiong
arXiv Machine Learning
Sep 18

Next-token functional estimation

The paper introduces a leave‑a‑window‑out estimator for next‑token functionals, such as the surprise probability and test error, in sequences of random variables. By deleting a window of length τ after each index, the estimator generalizes leave‑one‑out and achieves parametric error decay for stationary β‑mixing processes that admit a Marton coupling. The authors provide both upper bounds and a minimax lower bound for the surprise probability, and demonstrate through simulations that their method outperforms traditional baselines on Markov, moving‑average, and autoregressive processes.

By Milind Nakul, Vidya Muthukumar, Ashwin Pananjady
Hugging Face Trending Papers
Sep 17

Next-token functional estimation

Suppose we observe the first $n$ points of a sequence of random variables having length $n+1$, and wish to estimate a functional of the unobserved final point and the empirical measure of the $n$ obse...