The paper investigates how to efficiently repair stale key-value (KV) caches in retrieval‑augmented generation systems after document edits. It proposes a budgeted in‑place recomputation approach and evaluates training‑free position‑selection policies on a factual RAG benchmark. Across three model families, a contiguous edit‑local window consistently recovers most of the post‑edit answer quality while being 13–21 times faster than a full re‑prefill, though its effectiveness diminishes when answer‑bearing text moves downstream.
By Mingyang Mao, Wyatt Mackey, Xiaomin Lin
arXiv:2606. 01065v1 Announce Type: cross Abstract: Modern KV cache management assumes the chatbot workload: prompts arrive once and the cache grows append-only, so prefix caching and forward-only eviction are correct by construction.
By Bole Ma, Jan Eitzinger, Harald Koestler
arXiv:2609.26219v1 Announce Type: cross
Abstract: Long-running LLM agent workflows often revise interior context spans while retaining long suffixes. Although suffix tokens remain unchanged, altered...
By Guotao Yang, Rui Guo, Siwei He, Sheng Chen, Yitao Hu, Keqiu Li
arXiv:2609.36722v1 Announce Type: new
Abstract: Large language model (LLM) agents repeatedly load reusable content, such as skills, documents, and memory entries, into the current context. Re-encodin...
By Xinghao Chen, Junnan Dong, Cai Ke, Chak Tou Leong, Haocheng Sun, Keyu Chen, Siyu An, Ruizhi Qiao, Xing Sun, Wenjie Li, Xiaoyu Shen
CacheReforge is a method for recovering stale key‑value (KV) caches in large language models when lightweight adapters evolve. It represents stale caches as layer‑wise mixed‑version objects and uses adapter anchors, sensitivity calibration, drift accumulation, and restart boundaries to decide between direct reuse, bounded recomputation, or full suffix recovery. Experiments on Qwen2.5 models with continual LoRA updates show a 92.4% reduction in mean KL divergence while only recomputing 5.44% of layers and cutting cache‑maintenance time by 93.2% compared to full prefill.
By Yuhang Cao, Yanzhou Mu, Chunrong Fang, Zhenyu Chen
arXiv:2609.10266v1 Announce Type: new
Abstract: LLM serving systems already reuse KV caches, but only when the reused text sits at the very start of the prompt. Two growing workloads break this condi...
By Xi Shi, Qian Lou
arXiv:2609.25054v1 Announce Type: new
Abstract: For a long-horizon LLM agent, the memory question is not what was once recorded but what \emph{currently holds}. Most designs answer it only indirectly...
By Bowen Qin, Yao Lu
arXiv:2608. 11231v1 Announce Type: new Abstract: LLM serving is increasingly accelerated by position-independent caching (PIC).
By Yirui Liu, Ruoling Qi, Longwen Wang, Xuaner Wu, Jian Chen, Yuxin Jin, Jiawei Shao, Xuelong Li
arXiv:2609.24635v1 Announce Type: new
Abstract: When a language model reads an operation such as "Swap the contents of Box F and Box B", its forward pass writes keys and values for those tokens into...
By Lingfeng Wu, Behzad Shomali
arXiv:2608.30647v1 Announce Type: cross
Abstract: Language models can answer from precomputed memory, a model's saved reading of a body of material, reused across requests instead of read again at ea...
By Asa Shepard
arXiv:2606. 20737v2 Announce Type: replace Abstract: We study factual edit propagation in a controlled synthetic knowledge-graph QA setting using a 2x2 grid that crosses loop recurrence with shared-memory access: a dense transformer (Dense), a looped transformer (Loop), a dense backbone with shared memory (Dense+Mem), and a looped backbone with shared memory (loop-memory coupling, LMC).
By Yanan Niu
arXiv:2608. 07855v1 Announce Type: new Abstract: Multi-turn Reasoning-and-Acting (ReAct) agents accumulate growing trajectories of reasoning, tool calls, and observations.
By Weizhong Huang, Jinchao Zhang, Xiawu Zheng