arXiv Machine Learning July 21, 2026 By Felix Benning, Leif D\"oring

Gradient Span Algorithms Make Predictable Progress in High Dimension

Read the original on arXiv Machine Learning →

arXiv:2410. 09973v2 Announce Type: replace-cross Abstract: We prove that all 'gradient span algorithms' have asymptotically deterministic behavior on scaled Gaussian random functions as the dimension tends to infinity.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Aug 26

AdAdaGrad: Adaptive Batch Size Schemes for Adaptive Gradient Methods

arXiv:2402.11215v4 Announce Type: replace Abstract: The choice of batch size in minibatch stochastic gradient optimization is critical for both optimization and generalization performance in large-sc...

By Tim Tsz-Kit Lau, Han Liu, Mladen Kolar

arXiv Machine Learning

Jun 30

Geometrically Principled Randomized Optimization for Efficient LLM Training

arXiv:2510. 01878v2 Announce Type: replace Abstract: Low-rank gradient optimization for large language models is currently divided into two categories: structured methods that rigorously identify subspaces, and randomized approaches employed primarily for computational efficiency.

By Sahar Rajabi, Nayeema Nonta, Sirisha Rambhatla

llms benchmarks

arXiv Statistics ML

2d ago

Exact information accounting for SGD methods

arXiv:2610.00446v1 Announce Type: cross Abstract: As an alternative to the standard geometric analyses, we give an exact, information-theoretic analysis of stochastic gradient descent (SGD) and its v...

By Akshay Balsubramani

arXiv Machine Learning

Jun 2

Exploiting weight-space symmetries for approximating curvature

arXiv:2606. 00442v1 Announce Type: new Abstract: Many machine learning techniques rely on approximating a loss function's curvature, but this is notoriously hard to do at the scale of modern deep networks.

By Artem Artemev, Rui Xia, Benjamin M. Boyd, Youjing Yu, Felix Dangel, Guillaume Hennequin, Alberto Bernacchia

llms efficiency benchmarks

arXiv Machine Learning

Jun 30

Replica Symmetry Breaking and Algorithmic Thresholds in Empirical Risk Minimization under Multi-Index Model

arXiv:2606. 28573v1 Announce Type: new Abstract: Modern machine learning models are trained by optimizing high-dimensional non-convex empirical risk functions.

By Andrea Montanari, Kangjie Zhou

arXiv AI

Jun 29

How Width and Data Shape Generalization Scaling Laws in Quadratic Neural Networks

arXiv:2606. 28242v1 Announce Type: cross Abstract: Understanding how performance scales jointly with model size and data is a central problem in modern machine learning.

By Julius Girardin, Emanuele Troiani, Yizhou Xu, Vittorio Erba, Florent Krzakala, Lenka Zdeborov\'a

One email a morning, machine-written

Related stories

AdAdaGrad: Adaptive Batch Size Schemes for Adaptive Gradient Methods

Geometrically Principled Randomized Optimization for Efficient LLM Training

Exact information accounting for SGD methods

Exploiting weight-space symmetries for approximating curvature

Replica Symmetry Breaking and Algorithmic Thresholds in Empirical Risk Minimization under Multi-Index Model

How Width and Data Shape Generalization Scaling Laws in Quadratic Neural Networks