arXiv Machine Learning

Improved generalization bounds for binary linear classification via isoperimetry

arXiv:2505. 16713v3 Announce Type: replace-cross Abstract: We examine the concentration of uniform generalization errors around their expectation in binary linear classification problems via an isoperimetric argument.

arXiv Machine Learning
Jul 22

Fundamental limits of distributed multiclass classification from simple binary decisions

arXiv:2607. 19334v1 Announce Type: cross Abstract: We consider the problem of constructing a $K$-class classifier from the combination of $O(\log K)$ simple binary classifiers -- this is a natural paradigm to construct a sophisticated classifier in a distributed manner with each agent performing a relatively straightforward task.

By Ioannis Papageorgiou, Srinivas Nomula, Ayalvadi Ganesh, Sidharth Jaggi, Parimal Parag
arXiv Machine Learning
Sep 11

General Quantification of Covariate and Concept Shifts

arXiv:2609. 11918v1 Announce Type: new Abstract: Generalization under distribution shift remains a core challenge in modern machine learning, yet existing learning bound theory is limited to narrow, idealized settings and is non-estimable from samples.

By Hongbo Chen, Li Charlie Xia
arXiv Machine Learning
Jun 8

Stability beyond Bounded Differences: Sharp Generalization Bounds under Finite $L_p$ Moments

arXiv:2606. 06855v1 Announce Type: cross Abstract: While algorithmic stability is a central tool for understanding generalization of learning algorithms, existing high-probability guarantees typically rely on uniform boundedness or sub-Gaussian/sub-Weibull tail assumptions, which can be overly restrictive for modern settings with heavy-tailed or unbounded losses.

By Qianqian Lei, Soham Bonnerjee, Yuefeng Han, Wei Biao Wu
arXiv Machine Learning
Jul 30

Tight Generalization Bound for AdaBoost

arXiv:2607. 26838v1 Announce Type: new Abstract: In this paper we show that the generalization error of AdaBoost is $\Theta\big(\tfrac{d\ln(n\gamma^{2}/d)}{n\gamma^2}+\tfrac{\ln(1/\delta)}{n}\big)$, where $\gamma$ is the advantage guaranteed by the weak learner, $d$ is the VC-dimension of the class containing the weak hypotheses, $n$ is the sample size, and $\delta$ is the confidence parameter.

By Mikael M{\o}ller H{\o}gsgaard
arXiv Machine Learning
Jul 14

Exact Dynamics of Multi-class Stochastic Gradient Descent

arXiv:2510. 14074v2 Announce Type: replace-cross Abstract: We develop a framework for analyzing the learning dynamics of high-dimensional problems trained using one-pass stochastic gradient descent (SGD) with data from multiple anisotropic classes.

By Elizabeth Collins-Woodfin, Inbar Seroussi
arXiv Statistics ML
3d ago

Optimal Allocation and Volume under Surface

arXiv:2609.38875v1 Announce Type: cross Abstract: This paper develops a framework for estimation and inference on the volumes of sets that are projections of critical function sets, focusing particul...

By Kai Feng, Han Hong, Jessie Li, Wenshi Wei
arXiv Machine Learning
Sep 22

Statistical Inference for Adversarial Training: Central Limit Theorems via Optimal Transport

The paper rigorously analyzes the statistical and learning-theoretic properties of adversarial training models for classification, focusing on empirical optimal partial transport. It establishes two central limit theorems—one centered at the expected empirical value and another at the population value with smoothing—by leveraging the uniqueness of optimal potentials across various optimal transport formulations and empirical process theory. In the binary setting, the authors prove uniqueness of the optimal potential via a connection to multi-marginal optimal transport, and as additional results they derive stability of the saddle point, sample complexity, and concentration bounds for generalization error.

By Kim Jakwang, Kwon Dohyun