arXiv:2607. 21446v1 Announce Type: new Abstract: Post-training quantization (PTQ) of diffusion transformers (DiTs) to W4A4 severely degrades output quality, because activations entering each linear layer contain outliers that 4-bit formats cannot represent.
arXiv:2512. 00956v3 Announce Type: replace Abstract: Quantizing LLM weights and activations is a standard approach for efficient deployment, but a few extreme outliers can stretch the dynamic range and amplify low-bit quantization errors.
arXiv:2607. 02461v1 Announce Type: cross Abstract: Diffusion transformers (DiTs) achieve state-of-the-art image and video generation, but their multi-step sampling and growing parameter count make inference expensive.
PRQuant introduces a training‑free, low‑overhead method for low‑bit quantization of linear layers by permuting input channels that cause the largest quantization error into contiguous tail blocks and precomputing residual weight sub‑tensors. The approach eliminates the need for online gathering during inference, converting scattered residual compensation into a regular tail‑augmented GEMM and thereby reducing latency. Experiments show that PRQuant lowers down‑projection reconstruction error and outperforms standard MXFP4 and other post‑training quantization baselines on five downstream benchmarks, improving accuracy by up to 1.24 points on Qwen3‑4B‑Instruct‑2507.
arXiv:2609.00066v1 Announce Type: cross Abstract: NVFP4 is an efficient microscaling format for low-bit inference, but activation outliers can still degrade quantization accuracy within NVFP4 blocks....
arXiv:2601. 07475v2 Announce Type: replace-cross Abstract: The emergence of fine-grained numerical formats like NVFP4 presents new opportunities for efficient Large Language Model (LLM) inference.
arXiv:2601. 21626v2 Announce Type: replace-cross Abstract: Post Training Quantization (PTQ), a mainstream model compression technique, often leads to the paradoxical 'low error, high loss' phenomenon because it focuses solely on minimizing quantization error.
arXiv:2606. 26587v1 Announce Type: cross Abstract: Low-bit floating-point formats and semi-structured sparsity are increasingly supported by modern accelerators, yet combining them for LLM activation compression remains challenging: activations contain input-dependent outliers that dominate block scales in FP4 quantization, and directly applying N:M sparsity masks discards moderate values, coupling sparsification loss with quantization error.
arXiv:2605.11222v2 Announce Type: replace Abstract: Quantization is an effective strategy to reduce the storage and computation footprint of large language models (LLMs). Post-training quantization (...
arXiv:2505. 22988v3 Announce Type: replace-cross Abstract: The goal of quantization is to produce a compressed model whose output distribution is as close to the original model's as possible.
The paper revisits Kashin‑decomposition‑based weight quantization for large language models, introducing an improved algorithm that uses a sign‑randomized Discrete Cosine Transform (DCT) instead of a dense random orthogonal matrix. This change reduces per‑iteration cost from ≠(N^2) to ≠(N log N) and, combined with a greedy alternating‑update scheme, guarantees the four‑peak distribution needed for stable 2‑bit clustering while eliminating the need for multi‑restart k‑means. The resulting JAX pipeline, when paired with OPTQ‑style error compensation and QuIP‑style incoherence preprocessing, competes with state‑of‑the‑art quantization methods on OPT, Llama‑2, and Pythia at 4‑bit per channel, and remains numerically stable under stress configurations that cause other methods to diverge.
The paper surveys the use of linear, function‑preserving transforms in 4‑bit large‑language‑model (LLM) quantization, formalizing the underlying principle as the "Great Inversion"—the trade‑off between energy concentration favored by allocation‑flexible coding and within‑group flattening favored by grouped shared‑scale quantization. It reviews 200 works, classifies 43 transform methods by structure, data‑awareness, construction approach, and runtime cost, and examines how they interact with GPTQ rounding. The study also explores how different number formats (FP4, MXFP4, NVFP4) influence the optimal transform choice and outlines open research problems. "whyItMatters":"The survey clarifies the conflicting objectives in transform‑based LLM quantization and provides a practical guide for selecting transforms based on deployment regime, thereby informing future research and deployment strategies."
arXiv:2606. 12280v1 Announce Type: new Abstract: Post-training quantization lets large text-to-image diffusion transformers run on consumer GPUs, yet the hardware-specific trade-offs are seldom measured directly.