arXiv Machine Learning August 13, 2026 By Joao V. Cavalcanti, Ashia C. Wilson

SoftWater: Class-Aware Rate Allocation for Softmax Quantization

Read the original on arXiv Machine Learning →

arXiv:2608. 12026v1 Announce Type: new Abstract: Post-training quantization pipelines routinely leave the softmax output layer in high precision.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jul 7

Variable Bit-width Quantization: Learning Per-Group Precision for "Bigger-but-Smaller" Language Models

arXiv:2607. 02893v1 Announce Type: new Abstract: Low-bit quantization shrinks language models but treats precision as a single global hyper-parameter: every weight uses the same bit-width.

By Hamish Ogilvy

llms efficiency

arXiv Machine Learning

Jul 14

RDQ: Residual Distribution Quantization for Large Language Models

arXiv:2607. 10137v1 Announce Type: new Abstract: Post-training quantization (PTQ) of large language models degrades sharply below 4-bit precision.

By Prateek Singh

llms efficiency benchmarks

arXiv Machine Learning

Jul 15

Saturation Makes Quantization Error Additive: A Coverage Model with a Certificate

arXiv:2607. 12266v1 Announce Type: new Abstract: Mixed-precision quantization must decide which parts of a model to keep at higher precision.

By Joshua Hill

efficiency

arXiv AI

Jun 4

Recover-LoRA for Aggressive Quantization: Reclaiming Accuracy in 2-Bit Language Models via Low-Rank Adaptation with Knowledge Distillation on Synthetic Data

arXiv:2606. 04238v1 Announce Type: cross Abstract: Aggressive weight quantization to 2-bit precision offers substantial throughput and memory gains for large language model (LLM) inference, but typically incurs severe accuracy degradation.

By Devleena Das, Rajeev Patwari, Elliott Delaye, Ashish Sirasao

llms fine-tuning efficiency benchmarks

arXiv Machine Learning

Jul 31

Flat Score, Amplified Failures: How the Error Budget Masks Damage in Quantized LLM Agents

arXiv:2607. 27275v1 Announce Type: new Abstract: Post-training quantization to 4-bit weights is widely reported to be nearly lossless.

By Jiwon Jang, Kisu Yang, Heuiseok Lim, Hyunwoo Park

llms agents efficiency benchmarks safety

arXiv AI

Jun 9

High-Rate Quantized Matrix Multiplication II

arXiv:2605. 13768v2 Announce Type: replace-cross Abstract: This is the second part of the work investigating quantized matrix multiplication (MatMul).

By Or Ordentlich, Yury Polyanskiy

llms efficiency

Related stories

Variable Bit-width Quantization: Learning Per-Group Precision for "Bigger-but-Smaller" Language Models

RDQ: Residual Distribution Quantization for Large Language Models

Saturation Makes Quantization Error Additive: A Coverage Model with a Certificate

Recover-LoRA for Aggressive Quantization: Reclaiming Accuracy in 2-Bit Language Models via Low-Rank Adaptation with Knowledge Distillation on Synthetic Data

Flat Score, Amplified Failures: How the Error Budget Masks Damage in Quantized LLM Agents

High-Rate Quantized Matrix Multiplication II