Hugging Face Trending Papers
Low-Discrepancy Dither for Quantized Recurrent State Caches
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv Machine Learning
4d ago
Low-Discrepancy Dither for Quantized Recurrent State Caches
The paper investigates rounding strategies for low‑precision recurrent state caches in Mamba‑style and hybrid language models. It shows that a deterministic golden‑ratio Weyl dither consistently yields quantized models closer to full precision than stochastic rounding, across various models, storage formats, and long decoding horizons, without extra cost. In contrast, round‑to‑nearest can appear effective in short tests but degrades over long generations due to error accumulation.
arXiv AI
6d ago
STEPQuant: When and Where Errors Matter in Delta-Rule Recurrent State Quantization
arXiv:2609.38169v1 Announce Type: cross Abstract: Linear attention replaces growing KV caches with fixed-size recurrent states, yet these persistent states can become a substantial memory bottleneck...
arXiv Machine Learning
Jun 29
RateQuant: Optimal Mixed-Precision KV Cache Quantization via Rate-Distortion Theory
arXiv:2605. 06675v2 Announce Type: replace Abstract: Large language models cache all previously computed key-value (KV) pairs during generation, and this KV cache grows linearly with sequence length, making it a primary memory bottleneck for serving.
arXiv Machine Learning
Aug 31
DAMP: Decay-Aware Mixed-Precision Recurrent-State Quantization
The paper introduces DAMP, a decay‑aware mixed‑precision quantization scheme for recurrent‑state representations in GDN and KDA language models. By identifying high‑risk channels through quantization‑error energy and decay persistence, DAMP stores these channels at higher precision while compressing the rest to INT8, achieving a 9.9‑bit average precision. Experiments on Qwen3.6‑35B and Kimi‑Linear‑48B show a 69.1% reduction in recurrent‑state storage, up to 2.01× faster state‑update kernels, and up to 10.9% lower full‑model TPOT while preserving accuracy close to the FP32 baseline.
arXiv AI
Jul 2
When Less is More: 8-bit Quantization Improves Continual Learning in Large Language Models
arXiv:2512. 18934v2 Announce Type: replace-cross Abstract: Catastrophic forgetting poses a fundamental challenge in continual learning, particularly when models are quantized for deployment efficiency.
arXiv AI
Aug 6
Recurrent Residual Quantization: A Progressive Multi-Precision Representation for LLMs
arXiv:2608. 04048v1 Announce Type: cross Abstract: Serving large language models (LLMs) under diverse deployment constraints requires flexible trade-offs between accuracy, memory footprint, and throughput.