arXiv Machine Learning
Aug 10
Quantization Damage Is Multiplicative, Not Additive
arXiv:2608. 06564v1 Announce Type: new Abstract: Quantization is how large language models are actually deployed, and below four bits it is known to hurt.
Which Decisions Low-Bit Quantization Breaks, and How to Predict Them
arXiv:2608. 06564v2 Announce Type: replace Abstract: Quantization is how large language models are actually deployed, and below four bits it hurts.
Related stories
arXiv Machine Learning
Jul 21
Half the Experts, All the Code: One-Shot Domain Pruning of Mixture-of-Experts LLMs for Coding
arXiv:2607. 16721v1 Announce Type: new Abstract: The strongest open-weight coding models are mixture-of-experts (MoE) networks: most of their size comes from large pools of "expert" subnetworks, of which only a few act on any token.
arXiv Machine Learning
Jul 31
Flat Score, Amplified Failures: How the Error Budget Masks Damage in Quantized LLM Agents
arXiv:2607. 27275v1 Announce Type: new Abstract: Post-training quantization to 4-bit weights is widely reported to be nearly lossless.
arXiv AI
Aug 3
When Bits Break Recourse: Counterfactual-Faithful Quantization
arXiv:2605. 17160v2 Announce Type: replace-cross Abstract: Model quantization is widely used to reduce memory, latency, and deployment cost, and is typically judged by whether predictive accuracy is preserved.
arXiv AI
Jun 30
The Joint Effect of Quantization and Sampling Temperature on LLM Safety Alignment: A Factorial Analysis
arXiv:2606. 29581v1 Announce Type: cross Abstract: Modern LLM deployments routinely compress models and raise sampling temperature to reduce cost, latency, or repetition, yet safety evaluations usually treat these choices as fixed implementation details.
arXiv Machine Learning
1d ago
Certifying Compressed Language Models: An Audit and a Statistical Toolkit
arXiv:2608. 15046v1 Announce Type: new Abstract: A fraction of a point of benchmark accuracy is the usual evidence that a compressed model is equivalent to its original.
arXiv Machine Learning
Jun 24
One Ruler: A Same-Hands Re-Evaluation of Bivariate Causal Direction on Tuebingen, with a Parameter-Free Compression Baseline
arXiv:2606. 23767v1 Announce Type: new Abstract: Headline accuracies on the Tuebingen cause-effect pairs are routinely compared across papers even though each is measured under its authors' own protocol -- different pair subsets, weightings, model-selection, and decision rates.
Hugging Face Trending Papers
Jun 28
The Joint Effect of Quantization and Sampling Temperature on LLM Safety Alignment: A Factorial Analysis
Modern LLM deployments routinely compress models and raise sampling temperature to reduce cost, latency, or repetition, yet safety evaluations usually treat these choices as fixed implementation details. This leaves a practical uncertainty: does a model that is safe at FP16 and greedy decoding remain safe after it is quantized and sampled stochastically, or do the two deployment knobs amplify one another?
arXiv Machine Learning
Jul 28
Bigger or Cheaper? Scale and Quantization Effects on Uncertainty Signals in Vision-Language Models Under Image Degradation
arXiv:2607. 24440v1 Announce Type: cross Abstract: Vision-language models (VLMs) deployed on consumer hardware must decide when to answer and when to defer, and that decision depends on having a confidence signal that tracks correctness.
arXiv Machine Learning
Jul 15
Saturation Makes Quantization Error Additive: A Coverage Model with a Certificate
arXiv:2607. 12266v1 Announce Type: new Abstract: Mixed-precision quantization must decide which parts of a model to keep at higher precision.
arXiv Machine Learning
Jun 8
Predictable Compression Failures: Order Sensitivity and Information Budgeting for Evidence-Grounded Binary Adjudication
arXiv:2509. 11208v3 Announce Type: replace-cross Abstract: Transformers used for evidence-grounded binary adjudication (e.
arXiv Machine Learning
Jun 19
Displacement Is Not Direction: Evaluating Fidelity Metrics for Quantized LLM Deployment
arXiv:2606. 19558v1 Announce Type: new Abstract: Fidelity metrics, such as per-token KL divergence (KLD) against a high-precision reference, are often used in practice as low-cost proxies for benchmark quality.