← Back to all news
arXiv Machine Learning By Ehsan Jokar

Transforms for LLM Quantization: The Great Inversion and Format Co-Design

Read the original on arXiv Machine Learning →

The paper surveys the use of linear, function‑preserving transforms in 4‑bit large‑language‑model (LLM) quantization, formalizing the underlying principle as the "Great Inversion"—the trade‑off between energy concentration favored by allocation‑flexible coding and within‑group flattening favored by grouped shared‑scale quantization. It reviews 200 works, classifies 43 transform methods by structure, data‑awareness, construction approach, and runtime cost, and examines how they interact with GPTQ rounding. The study also explores how different number formats (FP4, MXFP4, NVFP4) influence the optimal transform choice and outlines open research problems. "whyItMatters":"The survey clarifies the conflicting objectives in transform‑based LLM quantization and provides a practical guide for selecting transforms based on deployment regime, thereby informing future research and deployment strategies."

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Computation and Language
Sep 11

Structured Transforms for Low-Overhead Quantization of Language Models

The paper revisits Kashin‑decomposition‑based weight quantization for large language models, introducing an improved algorithm that uses a sign‑randomized Discrete Cosine Transform (DCT) instead of a dense random orthogonal matrix. This change reduces per‑iteration cost from ≠(N^2) to ≠(N log N) and, combined with a greedy alternating‑update scheme, guarantees the four‑peak distribution needed for stable 2‑bit clustering while eliminating the need for multi‑restart k‑means. The resulting JAX pipeline, when paired with OPTQ‑style error compensation and QuIP‑style incoherence preprocessing, competes with state‑of‑the‑art quantization methods on OPT, Llama‑2, and Pythia at 4‑bit per channel, and remains numerically stable under stress configurations that cause other methods to diverge.

By Daria Cherniuk, Alexander Rudikov, Boris Kashin, Ivan Oseledets
llmsfine-tuningefficiency
More like this →
arXiv Machine Learning
Jul 24

KroQuant: Kronecker-Structured Block Transforms for Efficient Post-Training Quantization of Diffusion Transformers

arXiv:2607. 21446v1 Announce Type: new Abstract: Post-training quantization (PTQ) of diffusion transformers (DiTs) to W4A4 severely degrades output quality, because activations entering each linear layer contain outliers that 4-bit formats cannot represent.

By Yann Bouquet, Alireza Khodamoradi, Kristof Denolf, Mathieu Salzmann
llmsdiffusionefficiency
More like this →
arXiv AI
Sep 4

HARP: Hadamard-Preconditioned Adaptive Rotation Processor for Extreme LLM Quantization

HARP (Hadamard‑Preconditioned Adaptive Rotation Processor) is a learnable, structured two‑sided orthogonal processor that replaces fixed randomized Hadamard transforms in post‑training quantization of large language models. By representing rotations as sparse butterfly‑like block‑orthogonal stages and supporting mixed‑radix schedules, HARP adapts the quantization basis to each layer and calibration distribution while maintaining full‑precision equivalence. Across 2–4‑bit settings on Llama models from 1B to 70B, HARP consistently improves perplexity, delivers the strongest zero‑shot gains at 2 bits, and preserves deployment efficiency—achieving 128 tokens per second on Llama 2 7B at 2 bits, roughly 90% of RHT throughput and over twice the speed of FP16.

By Artur Zagitov, Gleb Molodtsov, Aleksandr Beznosikov
llmsefficiency
More like this →