arXiv AI
Jun 9
High-Rate Quantized Matrix Multiplication II
arXiv:2605. 13768v2 Announce Type: replace-cross Abstract: This is the second part of the work investigating quantized matrix multiplication (MatMul).
Price of metric universality in vector quantization is at most 0.11 bit
arXiv:2602. 05790v2 Announce Type: replace-cross Abstract: Fast computation of a matrix product $W^\top X$ is a workhorse of modern LLMs.
Related stories
arXiv Machine Learning
Sep 10
A Nuclear-Norm Lower Bound for Dithered Scalar Quantization of Matrix Products
arXiv:2609.05641v1 Announce Type: cross Abstract: We consider the problem of minimizing error in quantized matrix multiplication $C=AB$. Scalar quantization of the factors introduces rounding errors...
arXiv Machine Learning
Aug 24
Query Efficient Structured Matrix Learning
arXiv:2507.19290v2 Announce Type: replace-cross Abstract: We study the problem of learning a structured approximation (low-rank, sparse, banded, etc.) to an unknown matrix $A$ given access to matrix-...
arXiv Machine Learning
Jun 2
GPTQ-intrinsic LoRA: A Near-optimal Algorithm for Low-precision Quantization with Low-rank Adaptation
arXiv:2606. 01412v1 Announce Type: new Abstract: Post-training quantization is widely used for compressing large neural networks, but aggressive low-bit quantization can significantly degrade model quality.
arXiv Machine Learning
Jun 3
WaterSIC: Information-Theoretically (Near) Optimal Linear Layer Quantization
arXiv:2603. 04956v2 Announce Type: replace Abstract: This paper considers the problem of converting a given dense linear layer to low precision.
arXiv Machine Learning
Sep 3
Exact Limits of Random Projections for Preserving Geometry: Distance Recovery, Nearest-Neighbor Rankings, and Covariance Shape in Gaussian Models
arXiv:2609. 02155v1 Announce Type: new Abstract: The Johnson-Lindenstrauss (JL) lemma guarantees that a random projection of $n$ points to $m=O(\varepsilon^{-2}\log n)$ dimensions preserves pairwise squared distances within relative error $\varepsilon$ with high probability, and this dimension order is asymptotically optimal.
arXiv AI
Jun 11
What Limits Does Quantization Place on Dense Top-$k$ Retrieval? A Theoretical Study
arXiv:2606. 11780v1 Announce Type: cross Abstract: We establish conditions for embedding a corpus of $N$ documents as $d$-dimensional vectors such that every $k$-subset $S \subseteq [N]$ is realizable as a result of top-$k$ retrieval by some query vector.
arXiv AI
Jun 10
Minimum Distortion Quantization with Specified Output Distribution
arXiv:2606. 10458v1 Announce Type: cross Abstract: We derive the optimal quantizer of a real-valued random variable $W$ with distribution $P_W$ such that 1) the distribution of the quantization output $X$ that can take $k$ values follows any specified distribution $P_X$ over $\{1,\ldots,k\}$, and 2) the minimum mean squared error (MMSE) of estimating $W$ from $X$ is minimized.
arXiv Machine Learning
Jul 21
Exploiting Low-Rank Objective Structure in Discrete Quadratic Optimization
arXiv:2602. 20376v3 Announce Type: replace-cross Abstract: We study the problem of maximizing a complex-valued quadratic form over the $K^{\text{th}}$ roots of unity.
arXiv Machine Learning
Aug 5
Algorithms with Polynomially-Improved Approximation Factors for the $2 \rightarrow q$ Norm, and Applications
arXiv:2605. 25303v3 Announce Type: replace-cross Abstract: The $2 \rightarrow q$ norm of a matrix $X \in \mathbb{R}^{n \times d}$ is defined as $\lVert X \rVert_{2 \rightarrow q} = \sup_{\lVert v \rVert_2 = 1} \lVert Xv \rVert_q$.
arXiv Machine Learning
Jun 4
Adalina: Adaptive Linear Approximation for the Shapley Value and Beyond
arXiv:2604. 08438v2 Announce Type: replace Abstract: The Shapley value, and its broader family of semi-values, has received much attention in various attribution problems.
Hugging Face Trending Papers
Aug 27
Hadamard Flattening and Gaussian Pooling Sketch for Least Squares with Coordinate-wise Guarantee
Randomized sketch-and-solve algorithms accelerate overconstrained $\ell_2$ regression by replacing the input with a smaller problem. Standard subspace embeddings guarantee that the cost of the regress...