arXiv Machine Learning
The Structure of Quantization Damage in LLMs: Why the Next Bit Should Be Spent Globally
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Aug 20
Compress and Forget: bitsandbytes Quantization Amplifies Proactive Interference in LLMs
The study investigates how post‑training quantization (PTQ) affects proactive interference (PI) in large language models. Using bitsandbytes, the authors compare FP16, INT8, and INT4/NF4 precision across three instruction‑tuned models and find that INT4 quantization markedly degrades accuracy under high interference, with INT8 also incurring a smaller penalty in two of the three models. The degradation is linked to increased same‑key intrusion errors and originates in the quantized transformer backbone rather than the output layer.
arXiv Machine Learning
Jul 31
Flat Score, Amplified Failures: How the Error Budget Masks Damage in Quantized LLM Agents
arXiv:2607. 27275v1 Announce Type: new Abstract: Post-training quantization to 4-bit weights is widely reported to be nearly lossless.
arXiv Machine Learning
Aug 10
Quantization Damage Is Multiplicative, Not Additive
arXiv:2608. 06564v1 Announce Type: new Abstract: Quantization is how large language models are actually deployed, and below four bits it is known to hurt.
Hugging Face Trending Papers
Aug 8
Quantization Degradation in Large Language Models: A Signal-Noise Perspective
Post-training quantization reduces the deployment cost of large language models, yet how severely a quantized model degrades is not determined by bit-width alone. We systematically study weight-only post-training quantization across bit-widths, quantization methods, model scales and downstream tasks on multiple model families.
arXiv AI
Aug 11
Quantization Degradation in Large Language Models: A Signal-Noise Perspective
arXiv:2608. 08188v1 Announce Type: new Abstract: Post-training quantization reduces the deployment cost of large language models, yet how severely a quantized model degrades is not determined by bit-width alone.
arXiv AI
Jul 10
The Illusion of Equivalency: Statistical Characterization of Quantization Effects in LLMs
arXiv:2607. 08734v1 Announce Type: new Abstract: Post-training quantization is widely used to deploy large language models in resource-constrained settings, yet its evaluation relies almost exclusively on accuracy and perplexity.