arXiv Machine Learning
CurveTQ: Rotation-Free Trellis Quantization of LLM Weights via Curvature-Weighted Search
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Sep 1
ADMM-Q: An Improved Hessian-based Weight Quantizer for Post-Training Quantization of Large Language Models
arXiv:2605.11222v2 Announce Type: replace Abstract: Quantization is an effective strategy to reduce the storage and computation footprint of large language models (LLMs). Post-training quantization (...
arXiv Computation and Language
Sep 11
Structured Transforms for Low-Overhead Quantization of Language Models
The paper revisits Kashin‑decomposition‑based weight quantization for large language models, introducing an improved algorithm that uses a sign‑randomized Discrete Cosine Transform (DCT) instead of a dense random orthogonal matrix. This change reduces per‑iteration cost from ≠(N^2) to ≠(N log N) and, combined with a greedy alternating‑update scheme, guarantees the four‑peak distribution needed for stable 2‑bit clustering while eliminating the need for multi‑restart k‑means. The resulting JAX pipeline, when paired with OPTQ‑style error compensation and QuIP‑style incoherence preprocessing, competes with state‑of‑the‑art quantization methods on OPT, Llama‑2, and Pythia at 4‑bit per channel, and remains numerically stable under stress configurations that cause other methods to diverge.
arXiv AI
Sep 2
QTEA: Ternary LLMs with Sparse Residual Salient Weight and By-Column Optimization
arXiv:2609.00224v1 Announce Type: cross Abstract: Weight-only post-training quantization (PTQ) can alleviate the computational burden of serving large language models (LLMs) at scale. However, existi...
arXiv Machine Learning
Jul 10
KronQ: LLM Quantization via Kronecker-Factored Hessian
arXiv:2607. 07964v1 Announce Type: new Abstract: Post-training quantization (PTQ) is a widely adopted technique for compressing large language models (LLMs) without retraining.
arXiv AI
Jun 18
HeRo-Q: A General Framework for Stable Low Bit Quantization via Hessian Conditioning
arXiv:2601. 21626v2 Announce Type: replace-cross Abstract: Post Training Quantization (PTQ), a mainstream model compression technique, often leads to the paradoxical 'low error, high loss' phenomenon because it focuses solely on minimizing quantization error.
arXiv Machine Learning
Oct 2
JARQ: Joint Alternating Refinement for Quantization
JARQ is a plug‑in refinement method for group‑wise post‑training quantizers that alternates a joint least‑squares fit of all group scales with bounded Babai proposals to move many codes together on the current grid. It solves a bilinear box‑constrained mixed‑integer least‑squares problem without backpropagation, preserves the host model’s bit width, groups, zero points, and inference cost, and improves accuracy across Llama‑2, Llama‑3, and Qwen models when paired with RTN, GPTQ, OmniQuant, and AWQ hosts. In practice, JARQ reduces perplexity in 90 of 96 comparisons, cuts three‑bit RTN perplexity by up to 36%, raises mean multiple‑choice accuracy in 23 of 24 configurations, and enhances QEP, QuaRot, and OJBKQ outputs, all within under a minute per 7B block.