The paper introduces the concept of stale binding, where large language models (LLMs) answer with outdated values despite having newer information in context. It presents Controlled In-Context Memory (CICM), a benchmark to track and test the use of updated information in conversations and agent logs. Experiments on open‑source models reveal an attention drift mechanism that favors old values, and the authors propose a simple attention‑redirecting intervention that largely corrects these errors without harming correct answers.
As preferences, goals, and facts change, LLM agents must use the current state while earlier versions remain in context. Yet they can answer with an old value of the same variable, a failure that we c...
The paper introduces the Unified Memory Agent (UMA), a system that builds a query‑agnostic external memory from a data stream and reuses it across multiple question‑answering sessions. UMA employs a single policy to manage a structured Memory Bank via CRUD operations and uses Task‑Stratified GRPO to supervise memory maintenance based on QA trajectory rewards. The authors also present Ledger‑QA, a benchmark for long‑horizon state tracking, and demonstrate that UMA outperforms other methods on test‑time learning and accurate‑retrieval tasks, with UMA‑Specialist further improving performance after task adaptation.
arXiv:2607. 07847v1 Announce Type: new Abstract: As large language models (LLMs) become increasingly capable, the next question is how can we enable models to continually learn?
arXiv:2608. 02515v1 Announce Type: cross Abstract: Long-running assistants and agents consume interaction streams that eventually outgrow the context.
arXiv:2607. 12893v1 Announce Type: new Abstract: Long-term memory has become a foundational capability for LLM-based agents that accompany users across extended, multi-session interactions.
arXiv:2609.40118v1 Announce Type: new Abstract: As LLM capabilities advance, agents are tackling increasingly complex tasks over longer horizons. Their growing interaction histories make memory compa...
The paper investigates whether computation from earlier problems can aid large language models (LLMs) in solving subsequent ones. Preliminary experiments show that retained conversation history can both improve and degrade later-turn accuracy. To address this, the authors propose STAIR, a lightweight module that stores key-value pairs from previous responses and redirects new queries to this bank, improving later-turn accuracy by up to 11.67 percentage points across several benchmarks.
arXiv:2608. 05326v1 Announce Type: new Abstract: Autoregressive large language model inference is increasingly constrained by the memory footprint of the Key-Value (KV) cache.
A game character should not have to reread its entire life before every conversation. For locally deployed language-model characters, however, revising a few memories can invalidate a long reusable pr...
The paper introduces the Weighted Memory Tree (WMT), a hierarchical memory system for large language model agents that organizes execution histories into tasks, subtasks, and actions while assigning each memory a dynamic retention score. Event-based updates and selection-based decay allow WMT to preserve useful information, fold completed trajectories, suppress low-utility content, and retain access to folded context. Experiments on GAIA-Text with Qwen3-8B, Gemma 4 E4B, and Llama-3.1-8B show that WMT improves accuracy by an average of 9.97 percentage points and reduces prompt-token usage by 32.8%, while also limiting the persistence of unreliable information.
arXiv:2608. 19652v1 Announce Type: new Abstract: As LLM-based agents are deployed for longer and higher-stakes tasks, their memory systems continue to have crucial gaps.
arXiv:2609.18935v1 Announce Type: new Abstract: A game character should not have to reread its entire life before every conversation. For locally deployed language-model characters, however, revising...