arXiv Machine Learning
Risk-Controlled KV-Cache Eviction: From Memory Budgets to Risk Targets
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Jul 24
Error Certificates for KV-Cache Eviction via Randomized Design
arXiv:2607. 21475v1 Announce Type: cross Abstract: Deterministic KV-cache eviction keeps the top-$k$ tokens under an importance score and deletes the rest.
Hugging Face Trending Papers
Jul 23
Error Certificates for KV-Cache Eviction via Randomized Design
Deterministic KV-cache eviction keeps the top-$k$ tokens under an importance score and deletes the rest. We prove that this design cannot know what it destroyed: evicted values can be altered so that everything the serving system retains is unchanged while the true attention-output error grows arbitrarily, so no serving-time estimator of that error is consistent.
arXiv Machine Learning
Sep 22
PAGE: Partition-Aware Gated KV-Cache Eviction
PAGE is a partition‑aware gated KV‑cache eviction method that reframes eviction as a per‑input admission decision. It uses a single label‑free scalar— the early‑to‑late drop in pairwise top‑k head agreement—to classify inputs into a capacity‑bound class (where eviction is catastrophic) and a dilution‑prone class (where eviction is safe or beneficial). By thresholding this drop, PAGE applies a base evictor only when necessary, reducing the harm rate in the capacity‑bound regime from 0.75 to 0.026 and achieving a 29× improvement across four models and benchmarks without retraining the evictor.
arXiv AI
Jul 22
CompilerKV: Risk-Adaptive KV Compression via Offline Experience Compilation
arXiv:2602. 08686v3 Announce Type: replace-cross Abstract: Prefill-only KV compression freezes a token subset at the end of prefill and decodes from it without further eviction.
arXiv AI
Jun 17
AnchorKV: Safety-Aware KV Cache Compression via Soft Penalty with a Refusal Anchor
arXiv:2606. 17872v1 Announce Type: cross Abstract: Large language models (LLMs) outperform earlier architectures on generative inference and long-context tasks, but their large size introduces significant challenges in memory usage, energy cost, and on-device deployment.
arXiv Computation and Language
Sep 10
KVShareArena: KV-Cache Reuse Across Contexts and Model Checkpoints
arXiv:2609.10266v1 Announce Type: new Abstract: LLM serving systems already reuse KV caches, but only when the reused text sits at the very start of the prompt. Two growing workloads break this condi...