The paper introduces a unified Hessian-guided framework for post-training quantization of large language models. It derives a Hessian-based approximation of output changes due to joint weight and activation quantization, enabling simultaneous optimization of quantization ranges, linear transformations, and weight rounding under a single output-sensitivity objective. The method remains practical for modern models and consistently outperforms existing transformation and rounding techniques in both channel-wise and block-wise settings.
The paper introduces a unified Hessian-guided framework for post‑training quantization of large language models. By deriving a Hessian‑based approximation of output changes, it jointly optimizes quantization ranges, linear transformations, and weight rounding under a single output‑sensitivity objective. The method remains practical for modern models and consistently outperforms existing transformation and rounding techniques in both channel‑wise and block‑wise settings.
By Mads Toftrup, Marco Federici, Boris van Breugel, Paul Whatmough, Markus Nagel
arXiv:2601. 21626v2 Announce Type: replace-cross Abstract: Post Training Quantization (PTQ), a mainstream model compression technique, often leads to the paradoxical 'low error, high loss' phenomenon because it focuses solely on minimizing quantization error.
By Jinhao Zhang, Yunquan Zhang, Zicheng yan, Boyang Zhang, Jun Sun, Daning Cheng
arXiv:2606. 10531v1 Announce Type: cross Abstract: Quantization-aware training (QAT) is essential for extremely low-bit large language models (LLMs).
By Haoyu Wang, Xingyu Yu, Haiyan Zhao, Fengxiang Wang, Xu Han
G$^2$PTQ is a post‑training quantization framework that improves large language models by combining first‑ and second‑order information in a globally supervised, block‑wise optimization. It refreshes gradient and Hessian estimates before each Transformer block and uses a trust‑region scaling mechanism to stabilize gradient steps, preventing exploding weight updates. The method achieves better alignment with full‑precision models and outperforms state‑of‑the‑art baselines across various model families and bit‑widths.
By Ruikang Liu, Haoli Bai, Yuxuan Sun, Qian Zhang, Wenzheng Cai, Yanqi Hao, Feiyu Wang, Weidong Zhong, Zhuang Wang, Tong Yang, Xiangsheng Zhou
arXiv:2606. 10890v1 Announce Type: cross Abstract: Post-training quantization (PTQ) compresses large language models by mapping weights to low-bit representations.
By Juan Amboage, Pablo Monteagudo-Lago, Ian Colbert, Giuseppe Franco, Nicholas Fraser