arXiv Machine Learning June 2, 2026 By Linkai Ma, Tingzhou Yu, Petros Drineas

Stochastic Rounding Increases Small Singular Values

Read the original on arXiv Machine Learning →

arXiv:2606. 00312v1 Announce Type: cross Abstract: Over the past half-dozen years, stochastic rounding (SR) has regained significant attention as a quantization scheme for low-precision floating-point arithmetic, with applications spanning numerical analysis and modern machine learning systems.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

efficiency

arXiv AI

Jun 4

Low-Rank Decay for Grokking in Scale-Invariant Transformers: A Spectral-Geometric View

arXiv:2606. 04405v1 Announce Type: cross Abstract: Modern Transformer architectures frequently employ normalization mechanisms such as RMSNorm and Query-Key Normalization, making parts of the model approximately scale-invariant with respect to weight magnitudes.

By Mingyu Li

llms

arXiv Machine Learning

Jun 4

Novel Aspects of IEEE SA P3109 Arithmetic Formats for Machine Learning

arXiv:2606. 04028v1 Announce Type: new Abstract: The IEEE P3109 draft standard defines a parameterized family of binary floating-point formats and associated operations, with a focus on facilitating machine learning.

By Andrew Fitzgibbon, Christoph M. Wintersteiger, Jeffrey Sarnoff

efficiency

arXiv Machine Learning

1d ago

Spectral Certificates and Projection-DPP Rounding for Determinantal MAP Selection

arXiv:2606. 19411v3 Announce Type: replace Abstract: Selecting a fixed-size subset that maximizes the determinant of a positive semidefinite kernel is the MAP problem for a size-constrained determinantal point process and the classical maximum-entropy sampling problem.

By Richard Yi Da Xu

arXiv AI

Jun 4

Model-Preserving Adaptive Rounding

arXiv:2505. 22988v3 Announce Type: replace-cross Abstract: The goal of quantization is to produce a compressed model whose output distribution is as close to the original model's as possible.

By Albert Tseng, Zhaofeng Sun, Christopher De Sa

efficiency

arXiv Machine Learning

Jul 14

Training Diagonal Linear Networks with Stochastic Sharpness-Aware Minimization

arXiv:2503. 11891v2 Announce Type: replace Abstract: We analyze the landscape and training dynamics of diagonal linear networks in a linear regression task, with the network parameters being perturbed by isotropic normal noise during training.

By Gabriel Clara, Sophie Langer, Johannes Schmidt-Hieber

arXiv Machine Learning

Jul 30

Low-Precision Training of Large Language Models: Methods, Challenges, and Opportunities

arXiv:2505. 01043v2 Announce Type: replace Abstract: Large language models (LLMs) have achieved impressive performance across various domains.

By Zhiwei Hao, Jianyuan Guo, Li Shen, Yong Luo, Han Hu, Guoxia Wang, Dianhai Yu, Yonggang Wen, Dacheng Tao

llms efficiency

Related stories

Low-Rank Decay for Grokking in Scale-Invariant Transformers: A Spectral-Geometric View

Novel Aspects of IEEE SA P3109 Arithmetic Formats for Machine Learning

Spectral Certificates and Projection-DPP Rounding for Determinantal MAP Selection

Model-Preserving Adaptive Rounding

Training Diagonal Linear Networks with Stochastic Sharpness-Aware Minimization

Low-Precision Training of Large Language Models: Methods, Challenges, and Opportunities