arXiv Machine Learning
Budgeted Cache Repair for Cross-Context KV-Cache Reuse
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Computation and Language
Sep 10
KVShareArena: KV-Cache Reuse Across Contexts and Model Checkpoints
arXiv:2609.10266v1 Announce Type: new Abstract: LLM serving systems already reuse KV caches, but only when the reused text sits at the very start of the prompt. Two growing workloads break this condi...
arXiv Machine Learning
Sep 29
CacheReforge: Bounded Recovery for Stale KV Caches under Evolving Adapters
CacheReforge is a method for recovering stale key‑value (KV) caches in large language models when lightweight adapters evolve. It represents stale caches as layer‑wise mixed‑version objects and uses adapter anchors, sensitivity calibration, drift accumulation, and restart boundaries to decide between direct reuse, bounded recomputation, or full suffix recovery. Experiments on Qwen2.5 models with continual LoRA updates show a 92.4% reduction in mean KL divergence while only recomputing 5.44% of layers and cutting cache‑maintenance time by 93.2% compared to full prefill.
arXiv Machine Learning
Aug 4
RestoreKV: Recovering Full-Cache Behavior Under Aggressive Query-Agnostic KV Cache Eviction
arXiv:2608. 01247v1 Announce Type: cross Abstract: Query-agnostic KV cache eviction compresses a context once and reuses the resulting cache for arbitrary future queries, but performance can collapse under tight budgets.
arXiv AI
Oct 15
Budgeted KV-Cache Repair for Evolving Contexts
arXiv:2610.17338v1 Announce Type: new Abstract: Language-model agents revise their working context as information becomes obsolete, but deleting text does not erase its influence on cached computatio...
Hugging Face Trending Papers
Sep 15
Shared-Prefix KV Reuse Across Standard LoRA Adapters: Quality and Serving Tradeoffs
A common small-model deployment runs one shared backbone with several LoRA specialists that answer over the same context. Serving them naively re-prefills that shared context once per specialist. We s...
Hugging Face Trending Papers
Aug 2
RestoreKV: Recovering Full-Cache Behavior Under Aggressive Query-Agnostic KV Cache Eviction
Query-agnostic KV cache eviction compresses a context once and reuses the resulting cache for arbitrary future queries, but performance can collapse under tight budgets. Existing methods primarily improve which original KV pairs are retained.