arXiv Machine Learning
Jul 22
Contraction-Gauge Preconditioning for Quantized Matrix Multiplication
arXiv:2607. 18745v1 Announce Type: new Abstract: We study low-precision computation of C=AB with both factors quantized.
Structure of the Circular-Dyadic Convolution Error
arXiv:2607. 15293v1 Announce Type: cross Abstract: Dyadic and circular convolution can both be computed in $O(N\log N)$ time using the Hadamard transform and the FFT-computed discrete Fourier transform (DFT), respectively.
Related stories
Hugging Face Trending Papers
2d ago
Hadamard Flattening and Gaussian Pooling Sketch for Least Squares with Coordinate-wise Guarantee
Randomized sketch-and-solve algorithms accelerate overconstrained $\ell_2$ regression by replacing the input with a smaller problem. Standard subspace embeddings guarantee that the cost of the regress...
arXiv Machine Learning
Aug 18
Spectral Certificates and Projection-DPP Rounding for Determinantal MAP Selection
arXiv:2606. 19411v3 Announce Type: replace Abstract: Selecting a fixed-size subset that maximizes the determinant of a positive semidefinite kernel is the MAP problem for a size-constrained determinantal point process and the classical maximum-entropy sampling problem.
arXiv Machine Learning
2d ago
Transforms for LLM Quantization: The Great Inversion and Format Co-Design
The paper surveys the use of linear, function‑preserving transforms in 4‑bit large‑language‑model (LLM) quantization, formalizing the underlying principle as the "Great Inversion"—the trade‑off between energy concentration favored by allocation‑flexible coding and within‑group flattening favored by grouped shared‑scale quantization. It reviews 200 works, classifies 43 transform methods by structure, data‑awareness, construction approach, and runtime cost, and examines how they interact with GPTQ rounding. The study also explores how different number formats (FP4, MXFP4, NVFP4) influence the optimal transform choice and outlines open research problems. "whyItMatters":"The survey clarifies the conflicting objectives in transform‑based LLM quantization and provides a practical guide for selecting transforms based on deployment regime, thereby informing future research and deployment strategies."
arXiv Machine Learning
Jun 15
Muon$^p$: Muon with Fractional Spectral Powers
arXiv:2606. 13867v1 Announce Type: new Abstract: Muon is an increasingly widely used optimizer that replaces a gradient $G=USV^\top$ with its polar factor $UV^\top$, thereby flattening the singular spectrum.
arXiv Machine Learning
Jun 10
Learning Doubly Sparse Explicitly Conditioned Transforms
arXiv:2606. 10975v1 Announce Type: new Abstract: Finding convenient spaces in which certain hypotheses regarding an assumed sparse structure of natural signals hold true has become a desirable result in recent research, its implications being reflected in areas such as data compression, noise reduction and feature extraction.
arXiv Machine Learning
Jun 2
WUSH: Near-Optimal Adaptive Transforms for LLM Quantization
arXiv:2512. 00956v3 Announce Type: replace Abstract: Quantizing LLM weights and activations is a standard approach for efficient deployment, but a few extreme outliers can stretch the dynamic range and amplify low-bit quantization errors.
arXiv Machine Learning
1d ago
Hadamard Flattening and Gaussian Pooling Sketch for Least Squares with Coordinate-wise Guarantee
arXiv:2608.26552v1 Announce Type: cross Abstract: Randomized sketch-and-solve algorithms accelerate overconstrained $\ell_2$ regression by replacing the input with a smaller problem. Standard subspac...
arXiv Machine Learning
Jul 14
Spectral Origins of the Self-Correction Blind Spot in Autoregressive Generation
arXiv:2607. 09803v1 Announce Type: new Abstract: Large autoregressive language models exhibit a self-correction blind spot: they reliably fix identical errors when attributed to an external source yet fail to fix the same errors in their own outputs.
arXiv Machine Learning
Jun 19
Effective Dimension Governs Generalization in Quantum Kernel Vision Models
arXiv:2606. 20183v1 Announce Type: new Abstract: Recent quantum vision models-quantum vision transformers and quantum convolutional networks-report two striking but unexplained empirical phenomena: (i) ansatze with more, or more uniformly distributed, entanglement generalize better, and (ii) injecting quantum noise can improve test accuracy rather than degrade it.
arXiv Machine Learning
Jun 2
Stochastic Rounding Increases Small Singular Values
arXiv:2606. 00312v1 Announce Type: cross Abstract: Over the past half-dozen years, stochastic rounding (SR) has regained significant attention as a quantization scheme for low-precision floating-point arithmetic, with applications spanning numerical analysis and modern machine learning systems.
arXiv Machine Learning
Jun 11
Bernstein-Schur Kernels: Random Features by Sketched Modulation and Radial Randomization
arXiv:2606. 11255v1 Announce Type: new Abstract: Bernstein--Schur kernels are products of a finite-feature kernel (one with an explicit finite-dimensional feature map) and a completely monotone shift-invariant kernel: nonstationary kernels that fall between the shift-invariant and dot-product templates random features usually exploit, so in general neither Bochner sampling nor polynomial sketching applies to the full kernel directly.