The paper investigates where fine‑tuned behaviors in language models are instantiated by patching the key‑value (KV) cache between a LoRA‑fine‑tuned model and its base version. Results show that certain behaviors, such as German responses or emergent misalignment, can be transferred or removed by swapping KV cache segments, and that the impact grows with the amount patched and is concentrated at a few pivotal positions like system‑prompt and chat‑template tokens.
arXiv:2606. 17872v1 Announce Type: cross Abstract: Large language models (LLMs) outperform earlier architectures on generative inference and long-context tasks, but their large size introduces significant challenges in memory usage, energy cost, and on-device deployment.
CacheReforge is a method for recovering stale key‑value (KV) caches in large language models when lightweight adapters evolve. It represents stale caches as layer‑wise mixed‑version objects and uses adapter anchors, sensitivity calibration, drift accumulation, and restart boundaries to decide between direct reuse, bounded recomputation, or full suffix recovery. Experiments on Qwen2.5 models with continual LoRA updates show a 92.4% reduction in mean KL divergence while only recomputing 5.44% of layers and cutting cache‑maintenance time by 93.2% compared to full prefill.
arXiv:2607. 19957v1 Announce Type: cross Abstract: Key-Value (KV) cache reduces inference latency in large language models (LLMs).
arXiv:2609.10266v1 Announce Type: new Abstract: LLM serving systems already reuse KV caches, but only when the reused text sits at the very start of the prompt. Two growing workloads break this condi...
The paper introduces Random Attention, a method that evicts KV cache entries uniformly at random within each attention head while preserving the prompt. Experiments on four models and six reasoning tasks show that this simple strategy matches the performance of the best existing eviction methods and achieves 32‑43% higher throughput in vLLM deployments. The authors explain that the prompt is the most fragile cache component and that reasoning traces are redundantly stored across text and attention heads, making a selection score unnecessary.
The paper introduces DCKV, a differentiable codec designed to compress the key-value (KV) cache used in hybrid language models. DCKV jointly optimizes its transform, quantizer, and entropy models—conditioned on prompt tokens and neighboring positions—directly against a frozen language model, balancing compression rate with minimal impact on output quality. Experiments on Qwen3.5-2B demonstrate that DCKV can reduce the storage requirement for a compressed token by over 55× compared to bf16, with a loss of no more than 1.8 points on long‑context benchmarks, and by over 100× with a loss of no more than 3.6 points.
The paper introduces Random Attention, a method that evicts KV cache entries uniformly at random within each attention head while preserving the prompt. It demonstrates that this simple strategy matches or surpasses more complex eviction schemes across four models and six reasoning tasks, achieving 32‑43% higher throughput in vLLM deployments. Experiments reveal that the prompt is the most fragile cache component and that redundancy in the reasoning trace across text and attention heads protects against random eviction, eliminating the need for a selection score.
arXiv:2604.13226v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) rely heavily on Key-Value (KV) caching to minimize inference latency. However, standard KV caches are context-de...
arXiv:2611.06397v1 Announce Type: new Abstract: Non-prefix KV-cache reuse encodes retrieved chunks independently and concatenates their caches. To find where a small repair budget should be spent, we...
arXiv:2607. 28069v2 Announce Type: replace Abstract: Long-context retrieval and agentic workloads repeatedly reuse the same documents under changing instructions, histories, and document orders.
arXiv:2610.32109v1 Announce Type: new Abstract: Language models can answer a question from a supplied document or from their parametric knowledge. When later asked where the answer came from, what de...
arXiv:2607. 10582v1 Announce Type: cross Abstract: Large language model (LLM) agents accumulate heterogeneous context, including system instructions, plans, user turns, retrieved documents, tool outputs, and intermediate reasoning, whose key-value (KV) cache can become a major memory bottleneck.