arXiv Machine Learning
Aug 10
Quantization Damage Is Multiplicative, Not Additive
arXiv:2608. 06564v1 Announce Type: new Abstract: Quantization is how large language models are actually deployed, and below four bits it is known to hurt.
Flat Score, Amplified Failures: How the Error Budget Masks Damage in Quantized LLM Agents
arXiv:2607. 27275v1 Announce Type: new Abstract: Post-training quantization to 4-bit weights is widely reported to be nearly lossless.
Related stories
arXiv Machine Learning
Jul 15
Saturation Makes Quantization Error Additive: A Coverage Model with a Certificate
arXiv:2607. 12266v1 Announce Type: new Abstract: Mixed-precision quantization must decide which parts of a model to keep at higher precision.
arXiv Machine Learning
Jul 21
Half the Experts, All the Code: One-Shot Domain Pruning of Mixture-of-Experts LLMs for Coding
arXiv:2607. 16721v1 Announce Type: new Abstract: The strongest open-weight coding models are mixture-of-experts (MoE) networks: most of their size comes from large pools of "expert" subnetworks, of which only a few act on any token.
arXiv Machine Learning
Jun 19
Displacement Is Not Direction: Evaluating Fidelity Metrics for Quantized LLM Deployment
arXiv:2606. 19558v1 Announce Type: new Abstract: Fidelity metrics, such as per-token KL divergence (KLD) against a high-precision reference, are often used in practice as low-cost proxies for benchmark quality.
arXiv AI
Jun 10
Alignment Collapse Under KV Cache Quantization: Diagnosis and Mitigation
arXiv:2606. 09864v1 Announce Type: cross Abstract: Key-value (KV) cache quantization is widely used to reduce Large Language Model (LLM) inference memory, yet existing evaluations solely focus on measuring perplexity and accuracy without assessing the safety impact.
arXiv Machine Learning
Aug 11
Which Decisions Low-Bit Quantization Breaks, and How to Predict Them
arXiv:2608. 06564v2 Announce Type: replace Abstract: Quantization is how large language models are actually deployed, and below four bits it hurts.
arXiv AI
Aug 3
When Bits Break Recourse: Counterfactual-Faithful Quantization
arXiv:2605. 17160v2 Announce Type: replace-cross Abstract: Model quantization is widely used to reduce memory, latency, and deployment cost, and is typically judged by whether predictive accuracy is preserved.
arXiv AI
Aug 3
WitCert: Sound Runtime Risk Observability and Gating for KV-Cache Quantization
arXiv:2607. 28699v1 Announce Type: cross Abstract: KV-cache quantization is validated today by offline benchmark averages; a deployed system cannot tell whether compression is damaging the request it is serving right now.
arXiv AI
Jun 8
Perplexity Can Miss SAE Feature Damage Under Quantization
arXiv:2606. 03002v2 Announce Type: replace-cross Abstract: Quantization is a standard path to deploying large language models, and quantized models are typically judged acceptable when perplexity or downstream accuracy remains close to the full-precision original.
arXiv AI
Jun 3
How Quantization Changes Interpretable Features: A Sparse Autoencoder Analysis of Language Models
arXiv:2606. 03002v1 Announce Type: cross Abstract: Quantization is a standard path to deploying large language models, and a quantized model is typically judged acceptable when its perplexity or downstream accuracy stays close to the full-precision original.
arXiv Machine Learning
Jul 14
Silent Failures in Quantized LLM Reasoning: A Taxonomy-Based Analysis of Hollow Convergence and Failure Mode Shifts
arXiv:2607. 09999v1 Announce Type: cross Abstract: We show that post-training quantization can silently alter how large language models reason even when task accuracy is preserved.
arXiv Machine Learning
6d ago
SoftWater: Class-Aware Rate Allocation for Softmax Quantization
arXiv:2608. 12026v1 Announce Type: new Abstract: Post-training quantization pipelines routinely leave the softmax output layer in high precision.