The Flow has not summarised this story yet — read it at arXiv Computation and Language.
VestigeKV is a new KV‑cache technique that uses a 64‑dimensional vestigial branch—originally a RoPE component repurposed during NoPE training—as a query‑independent eviction signal. By reading only 11 % of each cache row, the method partitions the cache into an attended tier (top‑m rows) and an archive tier (all other rows), which is GPU‑resident and never deleted. The approach achieves near‑perfect retrieval (1.00 at 8×, 0.92 at 32×) without any training, quantization, or changes to weights or kernels.
arXiv:2606. 01502v1 Announce Type: cross Abstract: Frontier LLMs increasingly decide what a query attends to with a sparse-attention indexer that picks a few KV-cache blocks per query: attention's unit is now a small, reusable chunk.
arXiv:2606. 09916v1 Announce Type: cross Abstract: Multi-turn LLM agents fan short queries into long trajectories of tool calls, search results, and intermediate reasoning.
The paper investigates how block‑diffusion language models can use a constant‑size cache to enable efficient parallel decoding. By employing sequence mixers that summarize completed blocks into a reusable state and a block‑causal training objective, the authors pretrain three 3B block‑diffusion denoisers (attention, Mamba, and hybrid) on 300 B tokens. The resulting state‑space cache remains O(1) in memory and latency regardless of context length, yielding significant speed‑up and memory savings compared to traditional attention‑based caches, especially at very long sequences.
Minima-KV introduces a retention‑preserving hierarchy for mixed‑format paged attention that keeps recent and protected anchor pages in FP8 while older pages are compressed into packed TQ3, allowing every live‑request page to remain addressable. The approach uses format‑specific kernels and a globally normalized online‑softmax merge to compute partial attention states, enabling direct heterogeneous decoding without a dense shadow cache. Experiments on Qwen3.6‑27B on a 96‑GB NVIDIA RTX PRO 6000 Blackwell GPU show 3.50× compression over BF16 and 1.75× over FP8, with minimal impact on performance across long‑context benchmarks.
arXiv:2608. 11231v1 Announce Type: new Abstract: LLM serving is increasingly accelerated by position-independent caching (PIC).