arXiv Machine Learning
WUSH-KV: KV Cache Quantization with Data-Adaptive Transforms
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Aug 17
KV Cache Compression Through the Lens of Transform Coding
arXiv:2608. 14191v1 Announce Type: new Abstract: The key-value (KV) cache stores information from past tokens and is a major memory bottleneck in long-context inference.
arXiv AI
Jul 16
NSNQuant: A Double Normalization Approach for Calibration-Free Low-Bit Vector Quantization of KV Cache
arXiv:2505. 18231v3 Announce Type: replace-cross Abstract: Large Language Model (LLM) inference is typically memory-intensive, especially when processing large batch sizes and long sequences, due to the large size of key-value (KV) cache.
arXiv AI
Aug 6
Spend Bits Where Queries Look: KV Cache Vector Quantization with Attention-Preserving Transforms
arXiv:2608. 04074v1 Announce Type: cross Abstract: Long-context LLM decoding reads the key-value (KV) cache at every step.
arXiv Machine Learning
Jul 2
GSRQ: Gain-Shape Residual Quantization for Sub-1-bit KV Cache
arXiv:2607. 01065v1 Announce Type: new Abstract: The deployment of Large Language Models (LLMs) with extended context windows is increasingly constrained by the linear growth of Key-Value (KV) cache memory.
arXiv Machine Learning
Jun 8
AAAC: Activation-Aware Adaptive Codebooks for 4-bit LLM Weight Quantization
arXiv:2605. 08692v2 Announce Type: replace Abstract: Post-training weight-only quantization to 4 bits is widely used to reduce the memory and compute costs of large language model inference.
arXiv Machine Learning
4d ago
QuantMLA: Function-Aligned Dual-Path Quantization for Low-Bit MLA KV Caching
arXiv:2609.36760v1 Announce Type: new Abstract: Multi-Head Latent Attention (MLA) enables expressive multi-head attention with compact caches for its content and decoupled RoPE paths, yet cache memor...