arXiv AI
Lightweight but Linguistically Fair? Quantisation Robustness across Bavarian, German, and English
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
Hugging Face Trending Papers
Aug 25
Quantization Effects on Bangla Language Understanding in Large Language Models: A Systematic Evaluation
Post-training quantization lowers the memory footprint of Large Language Models (LLMs) and speeds up inference, which is why it is now common for on-device deployment. Most of what we know about its e...
arXiv Machine Learning
Aug 20
Compress and Forget: bitsandbytes Quantization Amplifies Proactive Interference in LLMs
The study investigates how post‑training quantization (PTQ) affects proactive interference (PI) in large language models. Using bitsandbytes, the authors compare FP16, INT8, and INT4/NF4 precision across three instruction‑tuned models and find that INT4 quantization markedly degrades accuracy under high interference, with INT8 also incurring a smaller penalty in two of the three models. The degradation is linked to increased same‑key intrusion errors and originates in the quantized transformer backbone rather than the output layer.
arXiv Machine Learning
10h ago
A Comprehensive Evaluation on Quantization Techniques for Large Language Models
arXiv:2507.17417v4 Announce Type: replace Abstract: For large language models (LLMs), post-training quantization (PTQ) significantly reduces memory and computational overhead. Despite rapid advanceme...
Hugging Face Trending Papers
Jul 23
QuantiBias: Benchmarking Quantization-Induced Bias in LLMs
Almost every large language model that reaches a broad audience is quantized: trained in full precision, then compressed for efficiency. This step is assumed harmless and its safety is rarely re-checked.
arXiv AI
Jun 24
Data Scale, Not Latency, Shapes Cross-Lingual Encoder Transfer in Streaming ASR
arXiv:2606. 24169v1 Announce Type: new Abstract: Adapting a streaming speech recognition model to a new language requires choosing between two plausible warm starts: a multilingual (ML) encoder or an English-only (EN) encoder.
Hugging Face Trending Papers
Aug 12
Language-Conditional Dequantization: Recovering What Quantization Steals from Non-English Languages
Aggressive quantization disproportionately harms multilingual capability: in the sub-4B INT3 GPTQ regime, we measure 2-4x larger perplexity degradation on non-English languages than on English. We propose Language-Conditional Dequantization (LCD), a post-hoc method that attaches per-language rank-2 LoRA corrections to the linear layers of an already-quantized model, adding 0.