arXiv Machine Learning
PuzzleKV: Page-Wise Low-Rank Decomposition for KV Cache Compression
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
Hugging Face Trending Papers
Jun 23
CompressKV: Semantic-Retrieval-Guided KV-Cache Compression for Resource-Efficient Long-Context LLM Inference
Long-context large language model (LLM) inference is increasingly constrained by the memory footprint and decoding cost of key-value (KV) caches, limiting sustainable deployment on resource-constrained hardware. Existing KV cache eviction methods typically apply heuristic token scoring over all heads in GQA-based LLMs.
arXiv AI
Jun 24
CompressKV: Semantic-Retrieval-Guided KV-Cache Compression for Resource-Efficient Long-Context LLM Inference
arXiv:2606. 24467v1 Announce Type: new Abstract: Long-context large language model (LLM) inference is increasingly constrained by the memory footprint and decoding cost of key-value (KV) caches, limiting sustainable deployment on resource-constrained hardware.
arXiv Machine Learning
Jul 20
VarRate: Training-Free Variable-Rate KV Cache Compression for Long-Context LLMs
arXiv:2607. 15498v1 Announce Type: cross Abstract: The key-value (KV) cache is the main memory bottleneck in long-context large language model (LLM) inference.
arXiv AI
5d ago
KVBoost: Chunk-Level Key-Value Cache Reuse with Deviation-Guided Recomputation for Efficient Large Language Model Inference
arXiv:2608.21362v1 Announce Type: new Abstract: Transformer-based large language models (LLMs) incur high prefill latency because key-value (KV) tensors must be recomputed for each request. Existing...
arXiv AI
Jul 8
DepthWeave-KV: Token-Adaptive Cross-Layer Residual Factorization for Long-Context KV Cache Compression
arXiv:2607. 06523v1 Announce Type: new Abstract: Long-context language model inference is increasingly limited by the memory bandwidth and capacity required to store key-value caches, yet existing compression methods often apply uniform budgets across layers or tokens and degrade retrieval when lexical cues and semantic states require different preservation.
arXiv Computation and Language
2d ago
NestedKV: Nested Memory Routing for Long-Context KV Cache Compression
NestedKV is a training‑free key‑only KV cache compression technique for long‑context language models that uses global, block‑level, and sliding‑window key anchors to score tokens via multi‑time‑scale cosine anomaly. It combines these rankings with a head‑adaptive outer learner and surprise‑gated token routing, requiring no model modification or additional training. Experiments on Qwen3 and Llama‑3.2 across benchmarks such as RULER, LongBench, and MMLU‑Pro show that NestedKV outperforms existing methods when the retained cache is small, achieving up to 19‑point gains on RULER and LongBench at a 75% retention rate.