arXiv:2608. 12026v1 Announce Type: new Abstract: Post-training quantization pipelines routinely leave the softmax output layer in high precision.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.
arXiv:2607. 02893v1 Announce Type: new Abstract: Low-bit quantization shrinks language models but treats precision as a single global hyper-parameter: every weight uses the same bit-width.
The paper introduces a post‑training softmax reparameterization technique that selects a functionally equivalent output head before quantization. By subtracting a scalar multiple of the vocabulary‑row mean from each output row and tuning this coefficient via validation KL, the method preserves the full‑precision softmax distribution while enabling efficient W4 quantization. Experiments on seven heads show significant error reductions and latency improvements, with the approach remaining complementary to other quantization strategies and transferable across datasets.
arXiv:2607. 10137v1 Announce Type: new Abstract: Post-training quantization (PTQ) of large language models degrades sharply below 4-bit precision.
arXiv:2607. 12266v1 Announce Type: new Abstract: Mixed-precision quantization must decide which parts of a model to keep at higher precision.
arXiv:2606. 04238v1 Announce Type: cross Abstract: Aggressive weight quantization to 2-bit precision offers substantial throughput and memory gains for large language model (LLM) inference, but typically incurs severe accuracy degradation.
arXiv:2607. 27275v1 Announce Type: new Abstract: Post-training quantization to 4-bit weights is widely reported to be nearly lossless.