arXiv:2608. 18147v1 Announce Type: cross Abstract: Adaptive stochastic quantization (ASQ) is a recently introduced quantization approach that optimizes the Mean Squared Error (MSE) for a given input while preserving unbiasedness.
arXiv:2606. 00289v1 Announce Type: new Abstract: Quantization is a fundamental tool used to compress datasets, neural network weights, and memory usage in a range of computational tasks.
arXiv:2608.30384v1 Announce Type: new Abstract: By introducing RSLM (Rotated Scaled Lloyd-Max), a family of training-free vector quantization codecs compressing embeddings to 1--4 bits per dimension,...
Layerwise Error Attribution for Fast and Robust Mixed-Precision Post-Training Quantization proposes a new allocation method that assigns bits to layers under a global memory budget using a small calibration set. The method derives a layerwise probabilistic analysis separating propagated error from local perturbations, enabling a simple, solver‑free allocation algorithm that is robust to corrupted calibration data. Experiments on DRUNet denoising tasks and quantized diffusion models show significant speed‑ups (up to 2,570×) and improved performance, with PSNR gains of up to 7.5 dB under corrupted calibration.
arXiv:2505. 22988v3 Announce Type: replace-cross Abstract: The goal of quantization is to produce a compressed model whose output distribution is as close to the original model's as possible.
The paper examines weight-only post‑training quantization (PTQ) and finds that minimizing reconstruction loss does not always improve model performance, sometimes even harming it on calibration data. It shows that weights with low reconstruction loss on one distribution can perform worse when activation distributions shift. To address this, the authors introduce Distributionally Robust Quantization (DRQ), a refinement step that minimizes worst‑case reconstruction loss over a constrained set of activation distributions while keeping the quantization grid unchanged. Experiments demonstrate that DRQ improves models quantized by several PTQ methods across dense and mixture‑of‑experts large language models, offering better downstream performance without extra inference cost.
arXiv:2608. 07019v1 Announce Type: new Abstract: Post-training quantization (PTQ) is widely used to reduce the memory and computational cost of large language models.
arXiv:2602. 03120v2 Announce Type: replace-cross Abstract: Post-Training Quantization (PTQ) is essential for deploying Large Language Models (LLMs) on memory-constrained devices, yet it renders models static and difficult to fine-tune.
As large language model inference shifts toward lower precision, post-training quantization (PTQ) becomes increasingly brittle, making quantization-aware training (QAT) essential for preserving model...
arXiv:2608. 13966v1 Announce Type: new Abstract: As large language model inference shifts toward lower precision, post-training quantization (PTQ) becomes increasingly brittle, making quantization-aware training (QAT) essential for preserving model quality.
arXiv:2608.30908v1 Announce Type: new Abstract: Fine-tuning Low-bit models aims to adapt a quantized model while keeping the final deployed checkpoint in the same low-bit form. This setting is practi...
arXiv:2608. 05127v1 Announce Type: cross Abstract: Achieving local differential privacy in distributed optimization while maintaining low communication cost remains challenging.
arXiv:2602. 05367v3 Announce Type: replace Abstract: Efficient deployment of large language models (LLMs) requires extreme quantization, forcing a critical trade-off between low-bit efficiency and performance.