arXiv Machine Learning
SpectrumQuant: Spectrum Alignment for LLM Quantization
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Oct 3
ConQuR: Corner Aligned Activation Quantization via Optimized Rotations for LLMs
ConQuR introduces a lightweight post‑training rotation calibration for large language model activation quantization. By learning orthogonal rotations that align normalized activations with the corners of an inscribed hypercube, the method distributes activation energy evenly and can be updated online without storing activations. Experiments on Llama‑2 and Llama‑3 models (3B–70B) show competitive or improved perplexity and reasoning performance while avoiding costly training or large offline storage.
arXiv Machine Learning
Oct 15
PrismQuant: Optimal Null-Space Rotations for Grouped Quantizers
PrismQuant introduces a rotation framework that aligns the leading activation eigenspace with the constant group subspace of asymmetric grouped INT4 quantizers, optimizing the alignment via Ky Fan trace maximization and providing a closed‑form solution. The method employs compact Householder transformations for efficient, gradient‑free construction and applies a predictive range law to relate unaligned activation energy and group size to quantization variation. Experiments on large language models up to 70B parameters show PrismQuant achieving state‑of‑the‑art perplexity and accuracy, while delivering significant speedups and memory reductions in deployment scenarios.
arXiv AI
Sep 30
ThinQuant: Scalable Rotation Learning for Weight and Activation Quantization of LLMs
ThinQuant introduces efficient rotation learning for low‑bit weight and activation quantization of large language models by reducing calibration data through a geometric selection of activations and solving a lower‑dimensional optimization problem via an ADMM algorithm. The method achieves comparable or better quantization performance with dramatically fewer calibration points, completing rotation calibration for Llama‑3‑70B in under 12 minutes and for Llama‑3.1‑405B in just over 2 hours on a single GPU. ThinQuant outperforms existing gradient‑free approaches such as DartQuant and gradient‑based SpinQuant in both speed and perplexity metrics on WikiText‑2.
arXiv Machine Learning
Jul 24
GaugeQuant: Online Learning of Quantization-Optimal Bases from LLM Symmetries
arXiv:2607. 20757v1 Announce Type: new Abstract: Transformers are known to have internal continuous symmetries that leave outputs invariant, while modifying quantization.
arXiv Machine Learning
Sep 21
Understanding LLM Quantization through Activation-Guided Compensation and Orthogonal Residuals
arXiv:2609.21450v1 Announce Type: new Abstract: Post-training weight-activation quantization reduces the memory and inference costs of large language models, but aggressive W4A4 quantization remains...
arXiv Machine Learning
Oct 22
InfoQuant: Shaping Activation Distributions for Low-Bit LLM Quantization
InfoQuant is a train‑free method that reshapes activation distributions to be more compatible with low‑bit quantizers. It uses Peak Suppression Orthogonal Transformation (PSOT) and adaptive outlier‑token selection to reduce numerical range while maintaining dispersion, thereby lowering quantization error. Across several large language model families, InfoQuant outperforms existing post‑training quantization and end‑to‑end training baselines, preserving 97% of floating‑point accuracy on average and narrowing the LLaMA‑2 13B performance gap by 42%.