arXiv AI
STEPQuant: When and Where Errors Matter in Delta-Rule Recurrent State Quantization
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Aug 31
DAMP: Decay-Aware Mixed-Precision Recurrent-State Quantization
The paper introduces DAMP, a decay‑aware mixed‑precision quantization scheme for recurrent‑state representations in GDN and KDA language models. By identifying high‑risk channels through quantization‑error energy and decay persistence, DAMP stores these channels at higher precision while compressing the rest to INT8, achieving a 9.9‑bit average precision. Experiments on Qwen3.6‑35B and Kimi‑Linear‑48B show a 69.1% reduction in recurrent‑state storage, up to 2.01× faster state‑update kernels, and up to 10.9% lower full‑model TPOT while preserving accuracy close to the FP32 baseline.
arXiv AI
1d ago
LeapQuant: Efficient Linear Attention with Accurate Recurrent State Quantization
arXiv:2609.38166v1 Announce Type: cross Abstract: Recent LLMs increasingly adopt hybrid designs that replace standard attention with linear attention, such as Gated DeltaNet (GDN) and Kimi Delta Atte...
arXiv AI
Aug 6
Recurrent Residual Quantization: A Progressive Multi-Precision Representation for LLMs
arXiv:2608. 04048v1 Announce Type: cross Abstract: Serving large language models (LLMs) under diverse deployment constraints requires flexible trade-offs between accuracy, memory footprint, and throughput.
arXiv AI
Sep 17
HyQuant: Hybrid-Precision Quantization for LLM Attention
HyQuant introduces a hybrid-precision quantization framework for large language model (LLM) attention modules. It quantizes most attention states to low-bit formats while preserving a small set of vertical‑line tokens and local‑window states in full precision, guided by lightweight attention‑pattern signals. This design achieves near‑lossless accuracy across tasks while improving memory and hardware efficiency.
arXiv AI
Sep 1
DASC: Decay-Aware State Compression for Hybrid Linear-Attention Serving
arXiv:2608.30386v1 Announce Type: cross Abstract: Hybrid linear-attention architectures have recently scaled to large open-weight models, offering quality competitive with full attention while substa...
arXiv Machine Learning
Sep 1
SemKV: Semantic Mixed-Precision KV Cache Quantization Guided by the Quality Cliff for Long-Context LLM Inference
arXiv:2608.28911v1 Announce Type: new Abstract: The key-value (KV) cache is the dominant memory bottleneck of long-context large language model (LLM) inference, growing linearly with context length....