arXiv AI
When Does Compensating Inherited Quantization Error Help? Mean-Square Guarantees for Layer-Wise Quantization and Their Limits
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Jul 15
Saturation Makes Quantization Error Additive: A Coverage Model with a Certificate
arXiv:2607. 12266v1 Announce Type: new Abstract: Mixed-precision quantization must decide which parts of a model to keep at higher precision.
arXiv AI
Oct 21
Activation Sensitivity as a Unifying Principle for Post-Training Quantization
arXiv:2601.11663v2 Announce Type: replace-cross Abstract: Post-training quantization (PTQ) methods for large language models compress pretrained weights to low bit-widths without retraining by optimi...
arXiv AI
Jun 4
Model-Preserving Adaptive Rounding
arXiv:2505. 22988v3 Announce Type: replace-cross Abstract: The goal of quantization is to produce a compressed model whose output distribution is as close to the original model's as possible.
arXiv Machine Learning
Sep 10
Stability and Generalization of Straight-Through Estimators for Training Two-Layer Quantized Neural Networks
arXiv:2609.06430v1 Announce Type: new Abstract: We study the identity straight-through estimator (STE) for training a two-layer binary-activation network with hinge loss from the perspective of Stati...
Hugging Face Trending Papers
Oct 6
Align, Then Correct: Training-Free Two-Stage Low-Rank Compensation for Extremely Quantized Large Language Models
Low-rank quantization error compensation (LQEC) recovers the accuracy lost under aggressive weight quantization by attaching a closed-form rank-$r$ adapter beside each frozen quantized weight, without...
arXiv Machine Learning
Sep 11
Why Does Post-Training Quantization Work?
Post‑training quantization compresses large language models by storing weights at reduced precision, introducing errors into hidden states that could accumulate with depth. However, pretrained models accumulate far less hidden‑state error than randomly initialized ones, largely preserving downstream performance. The study identifies two key mechanisms: (1) each layer’s new error tends to oppose inherited error, partially canceling it, and (2) the LM‑head geometry preserves high‑rank token scores, mitigating output changes.