arXiv Machine Learning June 9, 2026 By Filip Kova\v{c}evi\'c, Hong Chang Ji, Denny Wu, Mahdi Soltanolkotabi, Marco Mondelli

Full-Batch Gradient Descent Outperforms One-Pass SGD: Sample Complexity Separation in Single-Index Learning

Read the original on arXiv Machine Learning →

arXiv:2602. 02431v2 Announce Type: replace-cross Abstract: It is folklore that reusing training data more than once can improve the statistical efficiency of gradient-based learning.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jun 26

From One-Pass SGD to Data Reuse: Mini-Batch Scaling Laws in Sketched Linear Regression

arXiv:2605. 24316v2 Announce Type: replace Abstract: Scaling laws provide compact descriptions of how prediction error varies with compute, model size, and data, but existing theory mainly treats single-sample SGD or full data reuse, leaving the role of mini-batching unclear.

By Ziyan Chen, Zhongzhu Zhou, Ding-Xuan Zhou

safety

arXiv Machine Learning

Jun 16

SILAGE: Memory-Efficient, Full-Gradient-Free Nonconvex Optimization for Nested Finite Sums

arXiv:2606. 15832v1 Announce Type: new Abstract: Empirical risk minimization on massive datasets naturally exhibits a nested double finite-sum structure, where $N=nm$ total samples are logically or physically partitioned into $n$ blocks of size $m$ (e.

By Igor Sokolov, Laurent Condat, Peter Richt\'arik

efficiency benchmarks

arXiv Statistics ML

2d ago

Exact information accounting for SGD methods

arXiv:2610.00446v1 Announce Type: cross Abstract: As an alternative to the standard geometric analyses, we give an exact, information-theoretic analysis of stochastic gradient descent (SGD) and its v...

By Akshay Balsubramani

arXiv Machine Learning

Jun 17

The Implicit Bias of Steepest Descent with Mini-batch Stochastic Gradient

arXiv:2602. 11557v2 Announce Type: replace Abstract: A variety of widely used optimization methods like SignSGD and Muon can be interpreted as instances of steepest descent under different norm-induced geometries.

By Jichu Li, Xuan Tang, Difan Zou

safety

arXiv Machine Learning

Aug 26

AdAdaGrad: Adaptive Batch Size Schemes for Adaptive Gradient Methods

arXiv:2402.11215v4 Announce Type: replace Abstract: The choice of batch size in minibatch stochastic gradient optimization is critical for both optimization and generalization performance in large-sc...

By Tim Tsz-Kit Lau, Han Liu, Mladen Kolar

arXiv Machine Learning

Jul 2

Homogenization of $\ell_2$-Adversarial Training in High-Dimensions: Exact Dynamics under Stochastic Gradient Descent

arXiv:2607. 00207v1 Announce Type: cross Abstract: We develop a framework for analyzing the learning dynamics of $\ell_2$-adversarial training of single-index models on Gaussian mixtures in the high-dimensional limit under streaming stochastic gradient descent (SGD).

By Fabrizzio Sabelli

safety

One email a morning, machine-written

Related stories

From One-Pass SGD to Data Reuse: Mini-Batch Scaling Laws in Sketched Linear Regression

SILAGE: Memory-Efficient, Full-Gradient-Free Nonconvex Optimization for Nested Finite Sums

Exact information accounting for SGD methods

The Implicit Bias of Steepest Descent with Mini-batch Stochastic Gradient

AdAdaGrad: Adaptive Batch Size Schemes for Adaptive Gradient Methods

Homogenization of $\ell_2$-Adversarial Training in High-Dimensions: Exact Dynamics under Stochastic Gradient Descent