arXiv AI
Jun 4
Model-Preserving Adaptive Rounding
arXiv:2505. 22988v3 Announce Type: replace-cross Abstract: The goal of quantization is to produce a compressed model whose output distribution is as close to the original model's as possible.
BaKron: Efficient Quantization with Kronecker-Factored Hessians
arXiv:2608. 06291v1 Announce Type: cross Abstract: We accelerate a family of algorithms for neural network quantization whose geometry is informed by any Kronecker-factored approximation of the Hessian.
Related stories
arXiv AI
Sep 15
WaterKron and FlipFlop Hessian: Information-Theoretically Grounded Quantization with Kronecker-factored Hessians
The paper introduces WaterKron, a method that integrates two-sided GPTQ with row- and column-dependent waterfilling scales and entropy coding for post‑training quantization. It derives a high‑rate distortion measure relative to the full Hessian, introducing a Kronecker‑Hessian mismatch factor Φ that quantifies the distortion penalty of using a Kronecker approximation. Minimizing Φ leads to a Gaussian covariance‑fitting problem solved via classical flip‑flop updates, yielding a FlipFlop Hessian that empirically improves KL divergence and perplexity compared to other Hessian choices.
arXiv Machine Learning
Sep 1
ADMM-Q: An Improved Hessian-based Weight Quantizer for Post-Training Quantization of Large Language Models
arXiv:2605.11222v2 Announce Type: replace Abstract: Quantization is an effective strategy to reduce the storage and computation footprint of large language models (LLMs). Post-training quantization (...
arXiv Machine Learning
Jul 10
KronQ: LLM Quantization via Kronecker-Factored Hessian
arXiv:2607. 07964v1 Announce Type: new Abstract: Post-training quantization (PTQ) is a widely adopted technique for compressing large language models (LLMs) without retraining.
arXiv Machine Learning
Jun 19
CAGE: Curvature-Aware Gradient Estimation For Accurate Quantization-Aware Training
arXiv:2510. 18784v3 Announce Type: replace Abstract: Despite significant work on low-bit quantization-aware training (QAT), there is still an accuracy gap between such techniques and native training.
arXiv Machine Learning
Jul 30
GPTQ-2D: Cubic-Time Two-Sided Adaptive Rounding
arXiv:2607. 27042v1 Announce Type: cross Abstract: Adaptive rounding methods such as GPTQ, or equivalently Babai's nearest plane algorithm, round a real matrix to integers under a quadratic metric.
arXiv AI
Jun 18
HeRo-Q: A General Framework for Stable Low Bit Quantization via Hessian Conditioning
arXiv:2601. 21626v2 Announce Type: replace-cross Abstract: Post Training Quantization (PTQ), a mainstream model compression technique, often leads to the paradoxical 'low error, high loss' phenomenon because it focuses solely on minimizing quantization error.
arXiv AI
4d ago
ThinQuant: Scalable Rotation Learning for Weight and Activation Quantization of LLMs
ThinQuant introduces efficient rotation learning for low‑bit weight and activation quantization of large language models by reducing calibration data through a geometric selection of activations and solving a lower‑dimensional optimization problem via an ADMM algorithm. The method achieves comparable or better quantization performance with dramatically fewer calibration points, completing rotation calibration for Llama‑3‑70B in under 12 minutes and for Llama‑3.1‑405B in just over 2 hours on a single GPU. ThinQuant outperforms existing gradient‑free approaches such as DartQuant and gradient‑based SpinQuant in both speed and perplexity metrics on WikiText‑2.
arXiv Machine Learning
Jul 22
Contraction-Gauge Preconditioning for Quantized Matrix Multiplication
arXiv:2607. 18745v1 Announce Type: new Abstract: We study low-precision computation of C=AB with both factors quantized.
arXiv Machine Learning
Aug 4
Simplified Quadratic Gradient: A Unified Framework Bridging Gradient Descent and Newton-Type Methods by Synthesizing Hessians and Gradients
arXiv:2209. 03282v5 Announce Type: replace-cross Abstract: Accelerating the convergence of second-order optimization, particularly Newton-type methods, remains a pivotal challenge in algorithmic research.
arXiv Machine Learning
Sep 23
Beyond Scalar Sensitivity: Activation-Aware Mixed-Precision LLM Quantization with Cross-Layer Refinement
arXiv:2609.25916v1 Announce Type: new Abstract: Mixed-precision weight quantization is commonly formulated as a Multiple-Choice Knapsack Problem (MCKP), yet existing solvers rely on scalar sensitivit...
arXiv Machine Learning
Jun 2
Inner Product Aware Quantization: Provably Fast, Accurate, and Adaptive Algorithms
arXiv:2606. 00289v1 Announce Type: new Abstract: Quantization is a fundamental tool used to compress datasets, neural network weights, and memory usage in a range of computational tasks.