The paper investigates whether a frozen transformer can use another model’s KV cache as a substitute for its own, finding that this approach fails because the reader only accepts caches within a narrow tolerable set. Experimental results show that none of eight substitution methods improve performance over a baseline, while a different strategy—adding the sender’s cache as extra context and only adapting queries—does outperform the baseline. The authors conclude that pursuing faithful cache substitution is misguided and that augmenting the reader’s own cache is the more promising direction.
arXiv:2607. 08032v1 Announce Type: new Abstract: Large language models, and the agents built on them, spend an ever-growing share of their compute and memory on remembering: caching attention keys and values, carrying long prompts, maintaining recurrent state, and storing what happened in previous turns and sessions.
The paper investigates how to efficiently share internal states between agents built from frozen language models. It proposes a shared pool approach where each agent publishes its internal state once, allowing peers to read it via a single mechanism, thereby avoiding the need for a dedicated transfer bridge for every agent pair. The study finds that augmenting the key‑value cache and replacing activations—rather than swapping them—yields better accuracy with minimal training, and that a single adapter can generalize across multiple publishers when their internal states are similar to the receiver’s.
The paper introduces Random Attention, a method that evicts KV cache entries uniformly at random within each attention head while preserving the prompt. Experiments on four models and six reasoning tasks show that this simple strategy matches the performance of the best existing eviction methods and achieves 32‑43% higher throughput in vLLM deployments. The authors explain that the prompt is the most fragile cache component and that reasoning traces are redundantly stored across text and attention heads, making a selection score unnecessary.
arXiv:2606. 09916v1 Announce Type: cross Abstract: Multi-turn LLM agents fan short queries into long trajectories of tool calls, search results, and intermediate reasoning.
arXiv:2608. 11218v1 Announce Type: new Abstract: Independent agents that reason in latent space can share computed state as key-value cache fragments rather than text.
arXiv:2610.14754v1 Announce Type: new Abstract: Hybrid language models interleave Gated DeltaNet (GDN) layers, whose recurrent memory has a fixed size, with full-attention layers, whose key-value (KV...
Page-EntroKV is a new KV‑cache eviction framework designed for grouped‑query attention (GQA) in long‑context autoregressive language models. It pools token importance scores across query heads within each physical group using sink‑isolated Renyi‑2 entropy, then projects these pooled scores onto page frames and performs eviction at the hardware tuple level. The authors prove theoretical guarantees on union overhead, budget preservation, and needle‑retention, and demonstrate on a Qwen2.5‑1.5B‑Instruct pilot that Page‑EntroKV achieves exact union overhead control, eliminates sink masquerade, and retains 100% of critical tokens compared to mean‑pooling baselines.
arXiv:2610.23290v1 Announce Type: new Abstract: Diffusion-based multimodal large language models (dMLLMs) decode by iterative denoising with bidirectional attention, which precludes the standard auto...
arXiv:2609.10266v1 Announce Type: new Abstract: LLM serving systems already reuse KV caches, but only when the reused text sits at the very start of the prompt. Two growing workloads break this condi...
TwinKV is a training‑free, attention‑free repair pass that identifies and swaps orphaned and redundant tokens in a KV cache, improving long‑context inference for small models. It works by detecting near‑duplicate keys and can be composed with existing eviction policies without altering their scoring rules. Experiments on Qwen3‑4B and Llama‑3.2‑1B across LongBench, LooGLE, RULER, and MMLU‑Pro show that TwinKV consistently improves performance for most configurations, especially at tighter compression ratios.
The paper investigates how short memory representations of large‑language‑model (LLM) group discussions affect final decision quality. It introduces CrossProfile, a benchmark of 480 hidden‑profile tasks that separates supporting, decoy, and irrelevant facts while keeping their wording constant. Experiments with Qwen2.5‑7B, Llama‑3.1‑8B, and DeepSeek models show that heuristics based on rarity or relevance both fail in predictable ways, and that the decision model’s social‑proof bias further degrades accuracy. Removing attribution tags or randomizing them improves performance, yet natural transcripts still lose more evidence than no notes at all.
arXiv:2609.36722v1 Announce Type: new Abstract: Large language model (LLM) agents repeatedly load reusable content, such as skills, documents, and memory entries, into the current context. Re-encodin...