arXiv Machine Learning
Jun 18
Smoothness-Based Derandomization of PAC-Bayes Bounds
arXiv:2606. 19105v1 Announce Type: new Abstract: We study PAC-Bayes derandomization for smooth loss functions.
Smoothness-Based Derandomization of PAC-Bayes Bounds
We study PAC-Bayes derandomization for smooth loss functions. Our goal is to obtain generalization bounds that hold with high probability for deterministic predictors by exploiting smoothness properties of both the loss and the predictor class.
Related stories
arXiv Machine Learning
Jul 30
Minimax-Optimal Generalization Bounds for Smooth Deep Neural Networks Trained by (Stochastic) Gradient Descent
arXiv:2606. 06772v2 Announce Type: replace-cross Abstract: Characterizing the optimization dynamics and statistical performance of over-parameterized deep neural networks (DNNs) remains a central challenge in understanding the remarkable success of deep learning.
arXiv Machine Learning
Jun 19
Fisher-Geometric Sharpness and the Implicit Bias of SGD toward Flat Minima
arXiv:2606. 20469v1 Announce Type: new Abstract: A widely held intuition in deep learning is that stochastic gradient descent (SGD) implicitly favors flat minima and that flat minima generalize better, but standard Euclidean measures of flatness such as the trace or maximum eigenvalue of the loss Hessian are not invariant under reparametrizations that preserve the network function, which undermines the theoretical foundations of this narrative.
arXiv Machine Learning
Aug 13
Linear-Core Surrogates: Smooth Loss Functions with Linear Rates for Classification and Structured Prediction
arXiv:2604. 27742v2 Announce Type: replace Abstract: A fundamental dichotomy in the theory of classification sets smoothness against statistical efficiency: smooth surrogate losses such as the logistic loss enable fast $O(1/T)$ optimization but yield slow square-root $H$-consistency bounds, while piecewise-linear losses like the Hinge loss achieve optimal linear $H$-consistency rates but are non-differentiable.
arXiv Machine Learning
Jul 9
Bound to Disagree: Generalization Bounds via Certifiable Surrogates
arXiv:2602. 23128v2 Announce Type: replace Abstract: Generalization bounds for deep learning models are typically vacuous, not computable or restricted to specific model classes.
arXiv Machine Learning
Jul 7
Non-asymptotic Convergence of Stochastic Gradient Descent in Score-based Generative Models
arXiv:2607. 04775v1 Announce Type: cross Abstract: Score-based Generative Models (SGMs) have achieved impressive performance in data generation across a wide range of applications.
arXiv Statistics ML
2d ago
Exact information accounting for SGD methods
arXiv:2610.00446v1 Announce Type: cross Abstract: As an alternative to the standard geometric analyses, we give an exact, information-theoretic analysis of stochastic gradient descent (SGD) and its v...
arXiv Machine Learning
Sep 10
Understanding Uncertainty Sampling via Equivalent Loss
arXiv:2307.02719v5 Announce Type: replace Abstract: Uncertainty sampling is a classical active-learning strategy, yet the statistical objective induced by its query rule is often implicit. We introdu...
arXiv Machine Learning
Jun 19
A Solver-Free Training Method for Predict-then-Optimize
arXiv:2606. 19587v1 Announce Type: cross Abstract: We propose a scalable method for training prediction (machine learning) models in the predict-then-optimize paradigm, where model outputs serve as coefficients for a subsequent linear optimization task.
arXiv Machine Learning
2d ago
Can Domain Generalization be Guaranteed in Small-Sample Learning?
arXiv:2609.39512v1 Announce Type: new Abstract: The small-sample learning problem remains a fundamental challenge in machine learning because limited training data lead to unstable model estimation a...
Hugging Face Trending Papers
Jul 6
Non-asymptotic Convergence of Stochastic Gradient Descent in Score-based Generative Models
Score-based Generative Models (SGMs) have achieved impressive performance in data generation across a wide range of applications. While the statistical properties of their sampling procedures are increasingly well understood, the optimization dynamics underlying their training remain less explored.
arXiv Machine Learning
Jun 25
Learning Non-Vacuous Generalization Bounds from Optimization
arXiv:2206. 04359v3 Announce Type: replace Abstract: One of the fundamental challenges in the deep learning community is to theoretically understand how well a deep neural network generalizes to unseen data.