arXiv Computation and Language
KVShareArena: KV-Cache Reuse Across Contexts and Model Checkpoints
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv Machine Learning
Sep 29
CacheReforge: Bounded Recovery for Stale KV Caches under Evolving Adapters
CacheReforge is a method for recovering stale key‑value (KV) caches in large language models when lightweight adapters evolve. It represents stale caches as layer‑wise mixed‑version objects and uses adapter anchors, sensitivity calibration, drift accumulation, and restart boundaries to decide between direct reuse, bounded recomputation, or full suffix recovery. Experiments on Qwen2.5 models with continual LoRA updates show a 92.4% reduction in mean KL divergence while only recomputing 5.44% of layers and cutting cache‑maintenance time by 93.2% compared to full prefill.
arXiv Machine Learning
Oct 5
Budgeted Cache Repair for Cross-Context KV-Cache Reuse
arXiv:2610.02233v1 Announce Type: cross Abstract: Cross-context KV-cache reuse predicts a shared segment's keys and values under a new prefix instead of recomputing them, and has been reported to do...
arXiv Machine Learning
Sep 29
Evaluating the accuracy of KV cache reuse techniques
The paper discusses position‑independent KV cache reuse, a technique designed to cut latency in retrieval‑augmented generation by reusing chunk‑level KV caches across prompts. It argues that current evaluation methods overstate the accuracy of such reuse because they do not accurately capture the loss of accuracy, and that existing datasets lack the necessary reuse dynamics for thorough testing. To remedy this, the authors propose a new evaluation methodology that unambiguously measures accuracy loss and introduce Boxoffice, a tool that programmatically creates datasets with challenging KV cache reuse patterns.
arXiv AI
Jun 2
Leyline: KV Cache Directives for Agentic Inference
arXiv:2606. 01065v1 Announce Type: cross Abstract: Modern KV cache management assumes the chatbot workload: prompts arrive once and the cache grows append-only, so prefix caching and forward-only eviction are correct by construction.
arXiv Computation and Language
Sep 30
ATTUNER: Recomputation-Free KV Cache Reuse via Query-Side Adaptation
arXiv:2609.36722v1 Announce Type: new Abstract: Large language model (LLM) agents repeatedly load reusable content, such as skills, documents, and memory entries, into the current context. Re-encodin...
arXiv AI
Oct 15
LatentRemedy: Efficient Remedy For Non-Prefix KV Cache Reuse in Multi-Agent Systems
LatentRemedy is a method designed to improve inference efficiency in multi‑agent systems that share non‑prefix KV caches. By inserting a small number of task‑conditioned latent tokens generated via an MLP, the target agent can better interpret reused context without recomputing the shared block. Experiments demonstrate that LatentRemedy consistently outperforms existing reuse baselines on conflicting tasks while keeping TTFT close to direct reuse as cache size grows.