arXiv AI
Jun 10
Optimal Post-Training Quantization Scales and Where to Find Them
arXiv:2606. 10890v1 Announce Type: cross Abstract: Post-training quantization (PTQ) compresses large language models by mapping weights to low-bit representations.
ReQuant: Fixed-Grid Discrete Refinement for Post-Training Quantization
arXiv:2608. 07019v1 Announce Type: new Abstract: Post-training quantization (PTQ) is widely used to reduce the memory and computational cost of large language models.
Related stories
arXiv Machine Learning
1d ago
SchurQuant: Groupwise Discrete Optimization for Layer-Wise LLM Quantization
arXiv:2608. 15567v1 Announce Type: new Abstract: Weight-only post-training quantization (PTQ) enables the deployment of large language models under tight memory budgets, but accuracy often collapses at 2-3 bits.
arXiv AI
Jun 10
LC-QAT: Data-Efficient 2-Bit QAT for LLMs via Linear-Constrained Vector Quantization
arXiv:2606. 10531v1 Announce Type: cross Abstract: Quantization-aware training (QAT) is essential for extremely low-bit large language models (LLMs).
arXiv AI
Jun 9
Joint Structural Pruning and Mixed-Precision Quantization for LLM Compression
arXiv:2606. 07819v1 Announce Type: new Abstract: Recently, the efficiency of Large Language Models (LLMs) deployment has become a critical concern in practical applications.
arXiv Machine Learning
Jun 8
AAAC: Activation-Aware Adaptive Codebooks for 4-bit LLM Weight Quantization
arXiv:2605. 08692v2 Announce Type: replace Abstract: Post-training weight-only quantization to 4 bits is widely used to reduce the memory and compute costs of large language model inference.
arXiv AI
Aug 6
Recurrent Residual Quantization: A Progressive Multi-Precision Representation for LLMs
arXiv:2608. 04048v1 Announce Type: cross Abstract: Serving large language models (LLMs) under diverse deployment constraints requires flexible trade-offs between accuracy, memory footprint, and throughput.
arXiv Machine Learning
2d ago
QUASAR: Lowering the Loss Floor of Quantization-Aware Training with Loss-Aware Reconstruction
arXiv:2608. 13966v1 Announce Type: new Abstract: As large language model inference shifts toward lower precision, post-training quantization (PTQ) becomes increasingly brittle, making quantization-aware training (QAT) essential for preserving model quality.
arXiv AI
Aug 11
From Sweep to Seam: Interleaved Cross-Block Post-Training Quantization
arXiv:2608. 09595v1 Announce Type: new Abstract: Compressing large language models to two bits or fewer is increasingly feasible through block-wise post-training quantization; cross-block variants reconstruct neighboring Transformer blocks within a moving window.
arXiv AI
Jun 16
RaBiT: Residual-Aware Binarization Training for Accurate and Efficient LLMs
arXiv:2602. 05367v3 Announce Type: replace Abstract: Efficient deployment of large language models (LLMs) requires extreme quantization, forcing a critical trade-off between low-bit efficiency and performance.
arXiv Machine Learning
Aug 12
ReRound: Reconstructive Rounding to Resolve Midpoint Ambiguity in Calibration-Free LLM Quantization
arXiv:2608. 11045v1 Announce Type: new Abstract: ReRound (Reconstructive Rounding) is a post-training quantization method that addresses the midpoint ambiguity inherent in standard round-to-nearest (RTN) schemes when quantizing weights near the centers of quantization intervals.
Hugging Face Trending Papers
Jun 11
TWLA: Achieving Ternary Weights and Low-Bit Activations for LLMs via Post-Training Quantization
Large language models (LLMs) exhibit exceptional general language processing capabilities, but their memory and compute costs hinder deployment. Ternarization has emerged as a promising compression technique, offering significant reductions in model size and inference complexity.
arXiv AI
Jun 12
TWLA: Achieving Ternary Weights and Low-Bit Activations for LLMs via Post-Training Quantization
arXiv:2606. 13054v1 Announce Type: cross Abstract: Large language models (LLMs) exhibit exceptional general language processing capabilities, but their memory and compute costs hinder deployment.