← Back to all news
arXiv Machine Learning By Samy Houache (IMB, UB), Yann Traonmilin (IMB, UB), Jean-Fran\c{c}ois Aujol (UB, IMB)

Layerwise Error Attribution for Fast and Robust Mixed-Precision Post-Training Quantization

Read the original on arXiv Machine Learning →

Layerwise Error Attribution for Fast and Robust Mixed-Precision Post-Training Quantization proposes a new allocation method that assigns bits to layers under a global memory budget using a small calibration set. The method derives a layerwise probabilistic analysis separating propagated error from local perturbations, enabling a simple, solver‑free allocation algorithm that is robust to corrupted calibration data. Experiments on DRUNet denoising tasks and quantized diffusion models show significant speed‑ups (up to 2,570×) and improved performance, with PSNR gains of up to 7.5 dB under corrupted calibration.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Jun 4

Model-Preserving Adaptive Rounding

arXiv:2505. 22988v3 Announce Type: replace-cross Abstract: The goal of quantization is to produce a compressed model whose output distribution is as close to the original model's as possible.

By Albert Tseng, Zhaofeng Sun, Christopher De Sa
efficiency
More like this →
arXiv AI
5d ago

When Lower Reconstruction Loss Hurts: Distributionally Robust Refinement for Low-Bit LLM Quantization

The paper examines weight-only post‑training quantization (PTQ) and finds that minimizing reconstruction loss does not always improve model performance, sometimes even harming it on calibration data. It shows that weights with low reconstruction loss on one distribution can perform worse when activation distributions shift. To address this, the authors introduce Distributionally Robust Quantization (DRQ), a refinement step that minimizes worst‑case reconstruction loss over a constrained set of activation distributions while keeping the quantization grid unchanged. Experiments demonstrate that DRQ improves models quantized by several PTQ methods across dense and mixture‑of‑experts large language models, offering better downstream performance without extra inference cost.

By Yanlong Zhao, Xiaoyuan Cheng, Huihang Liu, Baihua He, Xinyu Zhang, Harrison Bo Hua Zhu, Wenlong Chen, Li Zeng, Zhuo Sun
llmsefficiency
More like this →