arXiv AI
LoopQuant: Accurate and Efficient Post-Training Quantization for Looped Transformers
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
2d ago
LoopQ: Quantization for Looped Language Models
arXiv:2605.16343v2 Announce Type: replace-cross Abstract: Looped language models (LoopLMs) increase effective computational depth by repeatedly applying shared Transformer blocks without increasing t...
arXiv Machine Learning
Oct 8
ResidualQuant: KV Cache Quantization for Looped Transformers with 2-Bit Residuals
ResidualQuant introduces a 2‑bit residual quantization scheme for KV caches in looped Transformers, leveraging the similarity of KV states across loops. By using the final‑loop KV states as a reference and applying least‑square scaling, rotations, and loop‑wise mixed precision, the method achieves accurate quantization down to INT2. Experiments on various looped Transformer models show that ResidualQuant improves the accuracy‑memory tradeoff, retaining near BF16 accuracy while reducing KV storage by 80.7% and boosting decode throughput and batch size on an RTX 5090.
arXiv Machine Learning
Sep 25
FlashLoop: Fast and Memory-Efficient Looped Transformers via Lazy Updates
FlashLoop is a training‑free inference framework for Looped Transformers that reduces cross‑loop redundancy by employing token‑sparse updates, sparse attention, and KV‑residual quantization. It exploits observations that, as loops progress, state changes concentrate on a small token subset, attention differences are dominated by a sparse key subset, and KV residuals become amenable to low‑bit quantization. The method achieves lossless accuracy with up to 1.64× speedup and 6× KV‑cache memory reduction across several Looped Transformer models.
arXiv AI
Jun 12
TWLA: Achieving Ternary Weights and Low-Bit Activations for LLMs via Post-Training Quantization
arXiv:2606. 13054v1 Announce Type: cross Abstract: Large language models (LLMs) exhibit exceptional general language processing capabilities, but their memory and compute costs hinder deployment.
Hugging Face Trending Papers
Jun 11
TWLA: Achieving Ternary Weights and Low-Bit Activations for LLMs via Post-Training Quantization
Large language models (LLMs) exhibit exceptional general language processing capabilities, but their memory and compute costs hinder deployment. Ternarization has emerged as a promising compression technique, offering significant reductions in model size and inference complexity.
arXiv AI
Sep 30
LeapQuant: Efficient Linear Attention with Accurate Recurrent State Quantization
arXiv:2609.38166v1 Announce Type: cross Abstract: Recent LLMs increasingly adopt hybrid designs that replace standard attention with linear attention, such as Gated DeltaNet (GDN) and Kimi Delta Atte...