arXiv AI
Aug 13
VQ-bench: A Composable Vector Quantization Framework
arXiv:2608. 11240v1 Announce Type: new Abstract: Vector quantization is an old problem but has recently become central to AI infrastructure.
Quanto: a PyTorch quantization backend for Optimum
Related stories
arXiv Machine Learning
Jun 2
Inner Product Aware Quantization: Provably Fast, Accurate, and Adaptive Algorithms
arXiv:2606. 00289v1 Announce Type: new Abstract: Quantization is a fundamental tool used to compress datasets, neural network weights, and memory usage in a range of computational tasks.
arXiv Machine Learning
Jun 3
WaterSIC: Information-Theoretically (Near) Optimal Linear Layer Quantization
arXiv:2603. 04956v2 Announce Type: replace Abstract: This paper considers the problem of converting a given dense linear layer to low precision.
arXiv Machine Learning
Jun 19
CAGE: Curvature-Aware Gradient Estimation For Accurate Quantization-Aware Training
arXiv:2510. 18784v3 Announce Type: replace Abstract: Despite significant work on low-bit quantization-aware training (QAT), there is still an accuracy gap between such techniques and native training.
arXiv Machine Learning
Jul 10
KronQ: LLM Quantization via Kronecker-Factored Hessian
arXiv:2607. 07964v1 Announce Type: new Abstract: Post-training quantization (PTQ) is a widely adopted technique for compressing large language models (LLMs) without retraining.
arXiv Machine Learning
Jun 16
QuantKAN: A Unified Quantization Framework for Kolmogorov Arnold Networks
arXiv:2511. 18689v3 Announce Type: replace Abstract: Kolmogorov--Arnold Networks (KANs) replace linear weights with spline-based functions, offering strong expressivity but posing challenges for low-precision deployment due to heterogeneous parameter distributions.
arXiv AI
Jun 18
HeRo-Q: A General Framework for Stable Low Bit Quantization via Hessian Conditioning
arXiv:2601. 21626v2 Announce Type: replace-cross Abstract: Post Training Quantization (PTQ), a mainstream model compression technique, often leads to the paradoxical 'low error, high loss' phenomenon because it focuses solely on minimizing quantization error.
arXiv AI
Jun 4
Model-Preserving Adaptive Rounding
arXiv:2505. 22988v3 Announce Type: replace-cross Abstract: The goal of quantization is to produce a compressed model whose output distribution is as close to the original model's as possible.
arXiv Machine Learning
Jul 24
KroQuant: Kronecker-Structured Block Transforms for Efficient Post-Training Quantization of Diffusion Transformers
arXiv:2607. 21446v1 Announce Type: new Abstract: Post-training quantization (PTQ) of diffusion transformers (DiTs) to W4A4 severely degrades output quality, because activations entering each linear layer contain outliers that 4-bit formats cannot represent.
arXiv Machine Learning
Jul 7
DynamiQ: Accelerating Gradient Synchronization using Compressed Multi-hop All-reduce
arXiv:2602. 08923v2 Announce Type: replace Abstract: Multi-hop all-reduce is the de facto backbone of large model training.
arXiv AI
Jun 9
ScaleSweep: Accurate NVFP4 Post-Training Quantization of LLMs via Block Scale Initialization
arXiv:2606. 07618v1 Announce Type: cross Abstract: NVFP4 is a recently introduced hardware-supported FP4 format that improves the fidelity of 4-bit quantization through fine-grained block scales.
arXiv AI
Aug 10
ReQuant: Fixed-Grid Discrete Refinement for Post-Training Quantization
arXiv:2608. 07019v1 Announce Type: new Abstract: Post-training quantization (PTQ) is widely used to reduce the memory and computational cost of large language models.