Hugging Face Trending Papers
Divergence Timing and Cumulative Disagreement under KV-Cache Eviction
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv Machine Learning
Sep 16
Divergence Timing and Cumulative Disagreement under KV-Cache Eviction
The paper studies how evicting entries from a KV‑cache affects the token distributions used in autoregressive language models. It derives an exact decomposition showing that the expected fraction of mismatched tokens equals a first‑mismatch contribution plus a post‑divergence exposure term weighted by its mismatch rate. Experiments on Meta‑Llama‑3.1‑8B‑Instruct and Qwen2.5‑7B‑Instruct demonstrate that a 50% retention strategy (SnapKV) delays divergence and reduces its frequency compared to other retention schemes, while post‑divergence total variation remains high and dominates the overall mismatch gap.
arXiv Machine Learning
Jun 2
MomentKV: Closing the Directional Gap in KV Cache Eviction for Long-Context Inference
arXiv:2606. 01563v1 Announce Type: new Abstract: Autoregressive decoding in Transformer-based language models relies on the KV cache, whose memory footprint grows linearly with sequence length and becomes the primary bottleneck for long-context inference.
arXiv AI
Oct 2
How Divergence Becomes Decision Flips in Compressed Language Models
arXiv:2610.00694v1 Announce Type: cross Abstract: Compression reports summarize how far a compressed language model moved from the dense one, usually by a KL divergence; a deployment that relies on t...
arXiv AI
Oct 7
Monte Carlo Estimation for KV Cache Eviction
arXiv:2610.07643v1 Announce Type: cross Abstract: Most KV-cache eviction methods ask, in effect, which memory appeared important while reading the prompt? We instead ask, which memory will matter whi...
arXiv Machine Learning
Sep 30
Tail-Influence Sampling for CVaR Policy Evaluation
arXiv:2609.38096v1 Announce Type: new Abstract: Policies with similar mean returns can differ sharply in rare failures, yet estimating lower-tail conditional value-at-risk (CVaR) accurately can requi...
arXiv Machine Learning
Jul 31
Stage-Replay Divergence Follows the KV Cache: Fixed-Prefix Precision Controls and Bidirectional Cache Transplantation
arXiv:2607. 28495v1 Announce Type: new Abstract: Stage-replay diagnostics reconstruct intermediate token prefixes and treat fresh-prefill continuation as continuation from the decoder state that originally reached the prefix.