The Flow has not summarised this story yet — read it at arXiv Machine Learning.
arXiv:2609.07901v1 Announce Type: new Abstract: Weight quantization largely determines the economics of serving open-weight LLMs. Its costs are usually assessed with capability benchmarks, on which 4...
The paper investigates where post‑training quantization (PTQ) harms large language models (LLMs) and how to best allocate a limited precision budget. By causally raising each layer to 8‑bit precision across nine open‑weight models, the authors find that quantization damage is diffuse rather than concentrated in specific task circuits or weight statistics, and that globally refining quantization granularity outperforms selectively protecting the most recoverable layers. They also observe that the residual accuracy loss is budget‑limited and that peak recovery locations correlate with architecture within families but not across families.
arXiv:2608.20873v1 Announce Type: new Abstract: Every way of teaching a deployed language model something new -- full fine-tuning, adapter merging, model editing -- replaces the released checkpoint,...
arXiv:2608.28809v1 Announce Type: new Abstract: Extreme low-bit inference offers a route toward smaller models and constrained deployment. Ternary language models restrict weights to $\{-1,0,+1\}$, a...
arXiv:2607. 09999v1 Announce Type: cross Abstract: We show that post-training quantization can silently alter how large language models reason even when task accuracy is preserved.
The paper investigates when post‑training quantization (PTQ) calibration helps or hurts text embedders, focusing on the amount of calibration (dose) and whether to calibrate at all. Across three models and 161 calibration runs at 4‑bit weight quantization, the effect ranges from significant gains to catastrophic degradation, driven mainly by the functional displacement of the quantized model. A validation‑gated abstention rule is proposed that avoids harmful calibrations while capturing most benefits, highlighting the trade‑off between safety and opportunity.