arXiv Machine Learning
Jun 17
Price of metric universality in vector quantization is at most 0.11 bit
arXiv:2602. 05790v2 Announce Type: replace-cross Abstract: Fast computation of a matrix product $W^\top X$ is a workhorse of modern LLMs.
Minimum Distortion Quantization with Specified Output Distribution
arXiv:2606. 10458v1 Announce Type: cross Abstract: We derive the optimal quantizer of a real-valued random variable $W$ with distribution $P_W$ such that 1) the distribution of the quantization output $X$ that can take $k$ values follows any specified distribution $P_X$ over $\{1,\ldots,k\}$, and 2) the minimum mean squared error (MMSE) of estimating $W$ from $X$ is minimized.
Related stories
arXiv AI
Jun 9
High-Rate Quantized Matrix Multiplication II
arXiv:2605. 13768v2 Announce Type: replace-cross Abstract: This is the second part of the work investigating quantized matrix multiplication (MatMul).
arXiv Machine Learning
Jun 2
Robust Learning of a Group DRO Neuron
arXiv:2601. 18115v2 Announce Type: replace Abstract: We study the problem of learning a single neuron under standard squared loss in the presence of arbitrary label noise and group-level distributional shifts, for a broad family of covariate distributions.
arXiv Machine Learning
Jun 30
Universality of empirical risk minimization
arXiv:2202. 08832v3 Announce Type: replace-cross Abstract: We study a general class of optimization problems with decision variable $\boldsymbol{\Theta} \in \mathbb{R}^{p \times k}$ and cost function which is the sum of $n$ terms, each dependent on $\boldsymbol{\Theta}$ through the $k$-dimensional projection $\boldsymbol{\Theta}^\top \boldsymbol{x}_i$, where $\boldsymbol{x}_i$, $i \leq n$ are i.
arXiv Machine Learning
6d ago
SoftWater: Class-Aware Rate Allocation for Softmax Quantization
arXiv:2608. 12026v1 Announce Type: new Abstract: Post-training quantization pipelines routinely leave the softmax output layer in high precision.
arXiv Machine Learning
Jul 21
Rate-Distortion-Perception Theory: Redefining the Fundamental Limits of Information Representation
arXiv:2607. 17232v1 Announce Type: cross Abstract: Classical rate-distortion (RD) theory has long established the fundamental limits of lossy compression by quantifying the minimum number of bits required to represent a source under a prescribed distortion constraint.
arXiv Machine Learning
Jul 15
Saturation Makes Quantization Error Additive: A Coverage Model with a Certificate
arXiv:2607. 12266v1 Announce Type: new Abstract: Mixed-precision quantization must decide which parts of a model to keep at higher precision.
arXiv Machine Learning
Jul 31
Quantum Speedups for Stochastic Optimization with Heavy-Tailed Noise
arXiv:2607. 25492v2 Announce Type: replace Abstract: We study stochastic optimization with heavy-tailed gradient noise.
arXiv AI
Jun 11
Quantized Stochastic Primal-Dual Methods for Distributed Optimization under Relaxed Global Geometry
arXiv:2606. 11339v1 Announce Type: cross Abstract: We study distributed optimization with stochastic gradients and finite-bit communication modeled by random (unbiased) quantization.
arXiv Machine Learning
Jul 14
Generalized and Unified Equivalences between Hardness and Pseudoentropy
arXiv:2507. 05972v3 Announce Type: replace-cross Abstract: Pseudoentropy characterizations give quantitatively precise formulations of the relationship between computational hardness and computational randomness.
arXiv Machine Learning
Jun 18
How fast can you find a good hypothesis?
arXiv:2509. 03734v3 Announce Type: replace-cross Abstract: In the hypothesis selection problem, we are given sample and query access to finite set of candidate distributions (hypotheses), $\mathcal{H} = \{H_1, \ldots, H_n\}$, and samples from an unknown distribution $P$, both over a domain $\mathcal{X}$.
arXiv Machine Learning
Aug 11
Optimal Learning Under Tsybakov Noise
arXiv:2608. 08416v1 Announce Type: new Abstract: Probably Approximately Correct (PAC) learning [Val84] is a fundamental learning model that has been extensively investigated.