The paper introduces the concept of stale binding, where large language models (LLMs) answer with outdated values despite having newer information in context. It presents Controlled In-Context Memory (CICM), a benchmark to track and test the use of updated information in conversations and agent logs. Experiments on open‑source models reveal an attention drift mechanism that favors old values, and the authors propose a simple attention‑redirecting intervention that largely corrects these errors without harming correct answers.
By Junyu Guo, Yuchen Fang, Shangding Gu, Costas Spanos, James Demmel, Javad Lavaei
As preferences, goals, and facts change, LLM agents must use the current state while earlier versions remain in context. Yet they can answer with an old value of the same variable, a failure that we c...
arXiv:2607. 01002v1 Announce Type: cross Abstract: In long-context use, large language models frequently synthesize answers from the meaning of a relevant context span rather than literally copy-pasting them.
By Aryo Pradipta Gema, Beatrice Alex, Pasquale Minervini
The paper introduces Declarative Attention (DA), a protocol that lets language models explicitly declare which parts of their context to focus on during generation. By partitioning decoding into full-context, region-specific, and recent-output-only modes, the inference engine can skip large portions of the KV cache, dramatically reducing attended tokens. Experiments on 15 long-context tasks with off-the-shelf models show significant savings (52.0% and 31.1% reductions) with only modest accuracy drops that diminish as model size increases.
By Namgyu Ho, Huzama Ahmad, Woosung Koh, Se-Young Yun, Tal Schuster, Cicero Nogueira dos Santos
Hybrid language models combine attention with a fixed-size recurrent state, yet the distinct roles of each component are not well understood. The authors introduce two cache-level interventions—split-prefill and state-swap—to isolate the contributions of the KV cache (attention) and the recurrent state. Experiments on Qwen3.5 and Falcon-H1 show that exact retrieval depends almost entirely on attention, while output language and persona rely mainly on recurrence, with the state-swap intervention confirming that answers derive from the KV side and language from the recurrent side.
By Kirill Afendulev, Alexey Dontsov, Elena Tutubalina, Anton Korznikov
The paper investigates why Mamba models underperform Transformers on in-context retrieval tasks. It challenges the common belief that Mamba’s fixed-size recurrent state limits retrieval, showing that even when the query precedes the context, Mamba still lags behind Transformers. The study reveals that Mamba’s selection gates can strongly influence output, but they rely on relevance signals that only emerge gradually with depth, causing a retrieval bottleneck beyond state capacity.
By Nir Yarden, Amit Ben-Artzy, Roy Schwartz