arXiv AI
Jun 9
High-Rate Quantized Matrix Multiplication II
arXiv:2605. 13768v2 Announce Type: replace-cross Abstract: This is the second part of the work investigating quantized matrix multiplication (MatMul).
WaterSIC: Information-Theoretically (Near) Optimal Linear Layer Quantization
arXiv:2603. 04956v2 Announce Type: replace Abstract: This paper considers the problem of converting a given dense linear layer to low precision.
Related stories
arXiv AI
Jun 18
HeRo-Q: A General Framework for Stable Low Bit Quantization via Hessian Conditioning
arXiv:2601. 21626v2 Announce Type: replace-cross Abstract: Post Training Quantization (PTQ), a mainstream model compression technique, often leads to the paradoxical 'low error, high loss' phenomenon because it focuses solely on minimizing quantization error.
arXiv AI
6d ago
VQ-bench: A Composable Vector Quantization Framework
arXiv:2608. 11240v1 Announce Type: new Abstract: Vector quantization is an old problem but has recently become central to AI infrastructure.
arXiv AI
Jun 4
Model-Preserving Adaptive Rounding
arXiv:2505. 22988v3 Announce Type: replace-cross Abstract: The goal of quantization is to produce a compressed model whose output distribution is as close to the original model's as possible.
arXiv Machine Learning
Jul 10
KronQ: LLM Quantization via Kronecker-Factored Hessian
arXiv:2607. 07964v1 Announce Type: new Abstract: Post-training quantization (PTQ) is a widely adopted technique for compressing large language models (LLMs) without retraining.
arXiv AI
Jun 2
BitsMoE: Efficient Spectral Energy-Guided Bit Allocation for MoE LLM Quantization
arXiv:2606. 00079v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) large language models reduce per-token computation through sparse expert activation, but their deployment remains memory-intensive because all expert weights must be kept resident in memory.
arXiv AI
Jun 10
Optimal Post-Training Quantization Scales and Where to Find Them
arXiv:2606. 10890v1 Announce Type: cross Abstract: Post-training quantization (PTQ) compresses large language models by mapping weights to low-bit representations.
arXiv Machine Learning
Jun 16
NanoQuant: Efficient Sub-1-Bit Quantization of Large Language Models
arXiv:2602. 06694v3 Announce Type: replace Abstract: Weight-only quantization has become a standard approach for efficiently serving large language models (LLMs).
arXiv Machine Learning
Jun 17
Price of metric universality in vector quantization is at most 0.11 bit
arXiv:2602. 05790v2 Announce Type: replace-cross Abstract: Fast computation of a matrix product $W^\top X$ is a workhorse of modern LLMs.
arXiv AI
Jun 10
LC-QAT: Data-Efficient 2-Bit QAT for LLMs via Linear-Constrained Vector Quantization
arXiv:2606. 10531v1 Announce Type: cross Abstract: Quantization-aware training (QAT) is essential for extremely low-bit large language models (LLMs).
arXiv AI
Jun 4
LiftQuant: Continuous Bit-Width LLM via Dimensional Lifting and Projection
arXiv:2606. 04050v1 Announce Type: cross Abstract: Existing quantization methods are fundamentally limited by rigid, integer-based bit-widths (e.
arXiv Machine Learning
Jul 24
KroQuant: Kronecker-Structured Block Transforms for Efficient Post-Training Quantization of Diffusion Transformers
arXiv:2607. 21446v1 Announce Type: new Abstract: Post-training quantization (PTQ) of diffusion transformers (DiTs) to W4A4 severely degrades output quality, because activations entering each linear layer contain outliers that 4-bit formats cannot represent.