arXiv AI
Jun 9
High-Rate Quantized Matrix Multiplication II
arXiv:2605. 13768v2 Announce Type: replace-cross Abstract: This is the second part of the work investigating quantized matrix multiplication (MatMul).
Price of metric universality in vector quantization is at most 0.11 bit
arXiv:2602. 05790v2 Announce Type: replace-cross Abstract: Fast computation of a matrix product $W^\top X$ is a workhorse of modern LLMs.
Related stories
arXiv Machine Learning
Jun 2
GPTQ-intrinsic LoRA: A Near-optimal Algorithm for Low-precision Quantization with Low-rank Adaptation
arXiv:2606. 01412v1 Announce Type: new Abstract: Post-training quantization is widely used for compressing large neural networks, but aggressive low-bit quantization can significantly degrade model quality.
arXiv Machine Learning
Jun 3
WaterSIC: Information-Theoretically (Near) Optimal Linear Layer Quantization
arXiv:2603. 04956v2 Announce Type: replace Abstract: This paper considers the problem of converting a given dense linear layer to low precision.
arXiv AI
Jun 11
What Limits Does Quantization Place on Dense Top-$k$ Retrieval? A Theoretical Study
arXiv:2606. 11780v1 Announce Type: cross Abstract: We establish conditions for embedding a corpus of $N$ documents as $d$-dimensional vectors such that every $k$-subset $S \subseteq [N]$ is realizable as a result of top-$k$ retrieval by some query vector.
arXiv AI
Jun 10
Minimum Distortion Quantization with Specified Output Distribution
arXiv:2606. 10458v1 Announce Type: cross Abstract: We derive the optimal quantizer of a real-valued random variable $W$ with distribution $P_W$ such that 1) the distribution of the quantization output $X$ that can take $k$ values follows any specified distribution $P_X$ over $\{1,\ldots,k\}$, and 2) the minimum mean squared error (MMSE) of estimating $W$ from $X$ is minimized.
arXiv Machine Learning
Jul 21
Exploiting Low-Rank Objective Structure in Discrete Quadratic Optimization
arXiv:2602. 20376v3 Announce Type: replace-cross Abstract: We study the problem of maximizing a complex-valued quadratic form over the $K^{\text{th}}$ roots of unity.
arXiv Machine Learning
Aug 5
Algorithms with Polynomially-Improved Approximation Factors for the $2 \rightarrow q$ Norm, and Applications
arXiv:2605. 25303v3 Announce Type: replace-cross Abstract: The $2 \rightarrow q$ norm of a matrix $X \in \mathbb{R}^{n \times d}$ is defined as $\lVert X \rVert_{2 \rightarrow q} = \sup_{\lVert v \rVert_2 = 1} \lVert Xv \rVert_q$.
arXiv Machine Learning
Jun 4
Adalina: Adaptive Linear Approximation for the Shapley Value and Beyond
arXiv:2604. 08438v2 Announce Type: replace Abstract: The Shapley value, and its broader family of semi-values, has received much attention in various attribution problems.
arXiv Machine Learning
Jul 8
Leech Lattice Vector Quantization for Efficient LLM Compression
arXiv:2603. 11021v2 Announce Type: replace Abstract: Scalar quantization of large language models (LLMs) is fundamentally limited by information-theoretic bounds.
arXiv Machine Learning
Jul 22
Contraction-Gauge Preconditioning for Quantized Matrix Multiplication
arXiv:2607. 18745v1 Announce Type: new Abstract: We study low-precision computation of C=AB with both factors quantized.
arXiv AI
Jun 2
Information-Theoretic Lower Bounds for Bit-Constrained Stochastic Optimization via a Reduction to Compressed Gaussian Mean Estimation
arXiv:2606. 00703v1 Announce Type: cross Abstract: Low-precision pretraining (FP8, MXFP4, NVFP4) is now standard for frontier language models, yet the literature is almost entirely achievability -- algorithms and empirical scaling laws -- with no matching characterization of what is information-theoretically possible.
arXiv Machine Learning
Jul 28
Distance-Matrix Wasserstein Statistics for Scalable Gromov--Wasserstein Learning
arXiv:2605. 14981v2 Announce Type: replace Abstract: Gromov--Wasserstein (GW) distances compare graphs, shapes, and point clouds through internal distances, without requiring a common coordinate system.