← Back to all news
arXiv AI

KBBQ: A Predictive Noise Law and the Limits of Spectrum Flattening in FP4 Quantization

The paper presents a second‑order theory of quantization noise for matrix multiplication, characterizing quantization formats by the variance they assign to each element. It derives a closed‑form signal‑to‑noise‑ratio law for floating‑point rounding, introduces an upper bound κ* that cannot be exceeded by any function‑preserving linear transform, and proposes KBBQ—a method that parameterizes how closely a transform approaches this bound. Experiments on W4A4 across four base models and two FP4 formats show that KBBQ outperforms the previous state of the art without extra deployment‑time computation.

Related stories

arXiv Computation and Language
Sep 11

Structured Transforms for Low-Overhead Quantization of Language Models

The paper revisits Kashin‑decomposition‑based weight quantization for large language models, introducing an improved algorithm that uses a sign‑randomized Discrete Cosine Transform (DCT) instead of a dense random orthogonal matrix. This change reduces per‑iteration cost from ≠(N^2) to ≠(N log N) and, combined with a greedy alternating‑update scheme, guarantees the four‑peak distribution needed for stable 2‑bit clustering while eliminating the need for multi‑restart k‑means. The resulting JAX pipeline, when paired with OPTQ‑style error compensation and QuIP‑style incoherence preprocessing, competes with state‑of‑the‑art quantization methods on OPT, Llama‑2, and Pythia at 4‑bit per channel, and remains numerically stable under stress configurations that cause other methods to diverge.

By Daria Cherniuk, Alexander Rudikov, Boris Kashin, Ivan Oseledets
llmsfine-tuningefficiency
More like this →
arXiv Machine Learning
Jul 24

KroQuant: Kronecker-Structured Block Transforms for Efficient Post-Training Quantization of Diffusion Transformers

arXiv:2607. 21446v1 Announce Type: new Abstract: Post-training quantization (PTQ) of diffusion transformers (DiTs) to W4A4 severely degrades output quality, because activations entering each linear layer contain outliers that 4-bit formats cannot represent.

By Yann Bouquet, Alireza Khodamoradi, Kristof Denolf, Mathieu Salzmann
llmsdiffusionefficiency
More like this →
arXiv Machine Learning
Aug 27

Transforms for LLM Quantization: The Great Inversion and Format Co-Design

The paper surveys the use of linear, function‑preserving transforms in 4‑bit large‑language‑model (LLM) quantization, formalizing the underlying principle as the "Great Inversion"—the trade‑off between energy concentration favored by allocation‑flexible coding and within‑group flattening favored by grouped shared‑scale quantization. It reviews 200 works, classifies 43 transform methods by structure, data‑awareness, construction approach, and runtime cost, and examines how they interact with GPTQ rounding. The study also explores how different number formats (FP4, MXFP4, NVFP4) influence the optimal transform choice and outlines open research problems. "whyItMatters":"The survey clarifies the conflicting objectives in transform‑based LLM quantization and provides a practical guide for selecting transforms based on deployment regime, thereby informing future research and deployment strategies."

By Ehsan Jokar
llmsefficiency
More like this →
arXiv Machine Learning
Jun 2

Stochastic Rounding Increases Small Singular Values

arXiv:2606. 00312v1 Announce Type: cross Abstract: Over the past half-dozen years, stochastic rounding (SR) has regained significant attention as a quantization scheme for low-precision floating-point arithmetic, with applications spanning numerical analysis and modern machine learning systems.

By Linkai Ma, Tingzhou Yu, Petros Drineas
efficiency
More like this →