arXiv Machine Learning June 16, 2026 By Igor Sokolov, Laurent Condat, Peter Richt\'arik

SILAGE: Memory-Efficient, Full-Gradient-Free Nonconvex Optimization for Nested Finite Sums

Read the original on arXiv Machine Learning →

arXiv:2606. 15832v1 Announce Type: new Abstract: Empirical risk minimization on massive datasets naturally exhibits a nested double finite-sum structure, where $N=nm$ total samples are logically or physically partitioned into $n$ blocks of size $m$ (e.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jun 9

Full-Batch Gradient Descent Outperforms One-Pass SGD: Sample Complexity Separation in Single-Index Learning

arXiv:2602. 02431v2 Announce Type: replace-cross Abstract: It is folklore that reusing training data more than once can improve the statistical efficiency of gradient-based learning.

By Filip Kova\v{c}evi\'c, Hong Chang Ji, Denny Wu, Mahdi Soltanolkotabi, Marco Mondelli

arXiv Machine Learning

Aug 12

Convergence of Sign-based Random Reshuffling Algorithms for Nonconvex Optimization

arXiv:2310. 15976v4 Announce Type: replace Abstract: signSGD is attractive in nonconvex optimization because it communicates sign-valued rather than full-precision gradients.

By Zhen Qin, Zhishuai Liu, Pan Xu

safety

arXiv Machine Learning

6d ago

A New First-Order Meta-Learning Algorithm with Convergence Guarantees

arXiv:2409. 03682v2 Announce Type: replace Abstract: Learning new tasks by leveraging prior experience is a fundamental trait of intelligent systems.

By El Mahdi Chayti, Martin Jaggi

llms efficiency safety

arXiv Machine Learning

Jul 24

Memory-Computation Tradeoffs in Semi Amortized Parametric Optimization

arXiv:2607. 20769v1 Announce Type: new Abstract: Learning-enabled decision systems often use offline data or computation to reduce online compute cost.

By Shijie Pan, Agustin Castellano, Zeyu Shen, Enrique Mallada

arXiv AI

Jul 21

Reducing Per-Sample Harm in Stochastic Optimization

arXiv:2607. 16261v1 Announce Type: cross Abstract: Modern optimizers combine gradients from the current mini-batch with historical optimization state, such as momentum or adaptive moments.

By Apostolos Avranas

computer-vision benchmarks

arXiv Machine Learning

Jun 19

Spectral DPPs via NEPv: A Scalable Continuous Relaxation of Determinantal MAP for Diversity-Aware Data Selection

arXiv:2606. 19411v1 Announce Type: new Abstract: Selecting a small, diverse, high-quality subset from a massive pool of candidates is a recurring primitive in modern machine learning -- data curation and coreset selection for training and fine-tuning large models, active-learning batch acquisition, prompt and exemplar selection for in-context learning, retrieval diversification, and experimental design.

By Richard Yi Da Xu

llms rag fine-tuning benchmarks

Related stories

Full-Batch Gradient Descent Outperforms One-Pass SGD: Sample Complexity Separation in Single-Index Learning

Convergence of Sign-based Random Reshuffling Algorithms for Nonconvex Optimization

A New First-Order Meta-Learning Algorithm with Convergence Guarantees

Memory-Computation Tradeoffs in Semi Amortized Parametric Optimization

Reducing Per-Sample Harm in Stochastic Optimization

Spectral DPPs via NEPv: A Scalable Continuous Relaxation of Determinantal MAP for Diversity-Aware Data Selection