ConQuR introduces a lightweight post‑training rotation calibration for large language model activation quantization. By learning orthogonal rotations that align normalized activations with the corners of an inscribed hypercube, the method distributes activation energy evenly and can be updated online without storing activations. Experiments on Llama‑2 and Llama‑3 models (3B–70B) show competitive or improved perplexity and reasoning performance while avoiding costly training or large offline storage.
By Chayne Thrash, Ali Abbasi, Soheil Kolouri
PrismQuant introduces a rotation framework that aligns the leading activation eigenspace with the constant group subspace of asymmetric grouped INT4 quantizers, optimizing the alignment via Ky Fan trace maximization and providing a closed‑form solution. The method employs compact Householder transformations for efficient, gradient‑free construction and applies a predictive range law to relate unaligned activation energy and group size to quantization variation. Experiments on large language models up to 70B parameters show PrismQuant achieving state‑of‑the‑art perplexity and accuracy, while delivering significant speedups and memory reductions in deployment scenarios.
By Yanlong Chen, Yining Chen, Song Zhang, Amirhossein Habibian, Yawei Li
ThinQuant introduces efficient rotation learning for low‑bit weight and activation quantization of large language models by reducing calibration data through a geometric selection of activations and solving a lower‑dimensional optimization problem via an ADMM algorithm. The method achieves comparable or better quantization performance with dramatically fewer calibration points, completing rotation calibration for Llama‑3‑70B in under 12 minutes and for Llama‑3.1‑405B in just over 2 hours on a single GPU. ThinQuant outperforms existing gradient‑free approaches such as DartQuant and gradient‑based SpinQuant in both speed and perplexity metrics on WikiText‑2.
By Mehdi Makni, Ryan Lucas, Rahul Mazumder
arXiv:2607. 20757v1 Announce Type: new Abstract: Transformers are known to have internal continuous symmetries that leave outputs invariant, while modifying quantization.
By Miguel P. Bento, Jo\~ao Seabra
arXiv:2609.21450v1 Announce Type: new
Abstract: Post-training weight-activation quantization reduces the memory and inference costs of large language models, but aggressive W4A4 quantization remains...
By Yamato Narita, Issei Sato
InfoQuant is a train‑free method that reshapes activation distributions to be more compatible with low‑bit quantizers. It uses Peak Suppression Orthogonal Transformation (PSOT) and adaptive outlier‑token selection to reduce numerical range while maintaining dispersion, thereby lowering quantization error. Across several large language model families, InfoQuant outperforms existing post‑training quantization and end‑to‑end training baselines, preserving 97% of floating‑point accuracy on average and narrowing the LLaMA‑2 13B performance gap by 42%.
By Ke Li, Dong An, Xiaoling Zang, Can Ye, Liang Xie, Qibo Qiu, Chen Shen, Xiaofei He, Wenxiao Wang
The paper presents a comprehensive evaluation of post‑training quantization (PTQ) techniques for large language models (LLMs), addressing gaps in prior work by conducting fair, large‑scale experiments on text‑only LLMs. It decouples quantization into pre‑quantization transformation and error mitigation, systematically studies granularity, symmetry, and data formats (MXFP4, NVFP4), and finds that rotation‑based transformations and low‑rank compensation with GPTQ can improve performance, while finer granularity and asymmetric quantization benefit activations more than weights. The study highlights the importance of understanding component interactions and the impact of scaling factor formats on low‑bit quantization performance.
By Yutong Liu, Cairong Zhao, Guosheng Hu
HARP (Hadamard‑Preconditioned Adaptive Rotation Processor) is a learnable, structured two‑sided orthogonal processor that replaces fixed randomized Hadamard transforms in post‑training quantization of large language models. By representing rotations as sparse butterfly‑like block‑orthogonal stages and supporting mixed‑radix schedules, HARP adapts the quantization basis to each layer and calibration distribution while maintaining full‑precision equivalence. Across 2–4‑bit settings on Llama models from 1B to 70B, HARP consistently improves perplexity, delivers the strongest zero‑shot gains at 2 bits, and preserves deployment efficiency—achieving 128 tokens per second on Llama 2 7B at 2 bits, roughly 90% of RHT throughput and over twice the speed of FP16.
By Artur Zagitov, Gleb Molodtsov, Aleksandr Beznosikov
arXiv:2603. 05500v2 Announce Type: replace-cross Abstract: Efficient and stable training of large language models (LLMs) remains a core challenge in modern machine learning systems.
By Zeju Qiu, Lixin Liu, Adrian Weller, Han Shi, Weiyang Liu
arXiv:2606. 02288v1 Announce Type: new Abstract: Massive activation spikes in Large Language Models (LLMs) severely degrade quantization by stretching dynamic ranges.
By Yung-Chin Chen, Chung Peng Lee, Ze-Wei Liou, Naveen Verma
The paper introduces TORQUE, a framework that enhances quantization by jointly optimizing which coordinates to keep at high precision before and after applying uniform random rotations, all within a fixed bit budget. By preserving large input coordinates before rotation and the largest-magnitude coordinates after rotation, TORQUE reduces quantization error and allows efficient use of offline-optimized codebooks. The authors provide an error upper bound, prove that top‑k pre‑rotation retention is optimal for each k, and demonstrate improved accuracy‑storage tradeoffs in Gaussian models and practical tasks such as nearest‑neighbor retrieval, KV‑cache compression, and activation compression.
By Ran Ben Basat, Michael Mitzenmacher, Shay Vargaftik
arXiv:2606. 09927v1 Announce Type: cross Abstract: Post-training quantization (PTQ) is one of the most practical ways to reduce the serving cost of Large Language Models (LLMs), but activation quantization remains difficult because outlier-dominated channels lead to large quantization errors.
By Patrik Czak\'o, G\'abor Kert\'esz, S\'andor Sz\'en\'asi