arXiv Machine Learning
Fast Matrix Multiplication in fp8: Certified Coefficient Optimization and Measured Error
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
2d ago
Beyond Accuracy: Prefix-Invariant Realizations of Low-Precision Fast Matrix Multiplication
arXiv:2609.39816v1 Announce Type: new Abstract: Fast matrix multiplication saves multiplications through exact cancellation, but rounding sums that mix token rows can leave contributions from later t...
arXiv Machine Learning
Sep 1
Sharp Approximation Rates for Neural Networks with Affine Latent Parameterizations
arXiv:2608.31157v1 Announce Type: new Abstract: Many parameter-efficient methods generate the parameters of a large neural network from a low-dimensional latent representation. Given an architecture...
arXiv Machine Learning
Sep 10
Accelerated Frank-Wolfe Algorithms: Complementarity Conditions and Sparsity
arXiv:2511.02821v2 Announce Type: replace-cross Abstract: We develop new accelerated first-order algorithms in the Frank-Wolfe (FW) family for minimizing smooth convex functions over compact convex s...
arXiv Machine Learning
Sep 11
Toward a First-Principles Update Geometry for the Language-Model Head
The paper proposes a new update geometry for the language‑model head by treating the head and softmax as a single module and using Hilbert’s projective distance to measure functional change. It replaces the spectral norm with the Euclidean row diameter, derives a tractable RowNorm update rule, and demonstrates that RowNorm substantially reduces step diameters and Hilbert perturbations while only slightly increasing validation loss.
arXiv Machine Learning
Sep 1
OrScale: Orthogonalised Optimization with Layer-Wise Trust-Ratio Scaling
arXiv:2605.07815v2 Announce Type: replace Abstract: Muon fixes the \emph{direction} of every matrix-valued update at the polar factor of its momentum, while each layer's step \emph{magnitude} is addr...
arXiv Machine Learning
Sep 11
EGGROLL, Unrolled: Understanding and Improving Low-Rank Evolution Strategies at Scale
EGGROLL replaces dense Gaussian perturbations in evolution strategies with low‑rank Gaussian products, enabling practical optimization of large language models while maintaining exactness on quadratic objectives. The paper analyzes the mean update field, error bounds, and shows that rank‑one perturbations add only a small variance penalty compared to dense ES. A new leave‑one‑out estimator, LOO‑ROLL, further reduces estimator MSE and improves post‑training performance on transformer blocks and GSM8K accuracy.