arXiv Machine Learning
PatchKV: Efficient KV Cache Recovery for Dynamically Edited LLM Contexts
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Jun 16
KVEraser: Learning to Steer KV Cache for Efficient Localized Context Erasing
arXiv:2606. 17034v1 Announce Type: cross Abstract: Post-hoc context erasing over the KV cache is challenging because a local edit has a global consequence: once a span has been processed, its influence propagates into the cached states of all subsequent tokens.
arXiv AI
Oct 15
Budgeted KV-Cache Repair for Evolving Contexts
arXiv:2610.17338v1 Announce Type: new Abstract: Language-model agents revise their working context as information becomes obsolete, but deleting text does not erase its influence on cached computatio...
arXiv AI
Sep 17
Contiguity, Not Importance: Budgeted Repair of Stale KV Caches After Document Edits
The paper investigates how to efficiently repair stale key-value (KV) caches in retrieval‑augmented generation systems after document edits. It proposes a budgeted in‑place recomputation approach and evaluates training‑free position‑selection policies on a factual RAG benchmark. Across three model families, a contiguous edit‑local window consistently recovers most of the post‑edit answer quality while being 13–21 times faster than a full re‑prefill, though its effectiveness diminishes when answer‑bearing text moves downstream.
arXiv AI
Jun 17
Models Take Notes at Prefill: KV Cache Can Be Editable and Composable
arXiv:2606. 17107v1 Announce Type: cross Abstract: Prefix caching reuses prefill only across an exactly shared prefix, so one changed field invalidates the entire downstream cache.
arXiv AI
Aug 25
KVBoost: Chunk-Level Key-Value Cache Reuse with Deviation-Guided Recomputation for Efficient Large Language Model Inference
arXiv:2608.21362v1 Announce Type: new Abstract: Transformer-based large language models (LLMs) incur high prefill latency because key-value (KV) tensors must be recomputed for each request. Existing...
arXiv Machine Learning
Sep 29
CacheReforge: Bounded Recovery for Stale KV Caches under Evolving Adapters
CacheReforge is a method for recovering stale key‑value (KV) caches in large language models when lightweight adapters evolve. It represents stale caches as layer‑wise mixed‑version objects and uses adapter anchors, sensitivity calibration, drift accumulation, and restart boundaries to decide between direct reuse, bounded recomputation, or full suffix recovery. Experiments on Qwen2.5 models with continual LoRA updates show a 92.4% reduction in mean KL divergence while only recomputing 5.44% of layers and cutting cache‑maintenance time by 93.2% compared to full prefill.