arXiv:2606. 28876v2 Announce Type: replace-cross Abstract: We study memory-managed long-context attention: explicit bounded memory with a learned query-independent writer, lifecycle control, query-aware reading, calibrated sparse fallback, and frozen-LLM generation from raw evidence.
arXiv:2608.22215v1 Announce Type: new Abstract: Large language model (LLM) agents operate in dynamic environments where knowledge continuously evolves. Existing memory systems typically treat externa...
REMORY is a neural memory network that augments a textual summary of an agent’s history with a small set of soft memory tokens. These tokens, conditioned on the summary, enable a frozen large language model to approximate the continuation it would generate using the full history, effectively acting as a residual connection along the sequence dimension. Experiments on SummHay and long‑horizon agent benchmarks show that REMORY improves source attribution, maintains insight coverage, and achieves near full‑context performance while using only 5.2% of the input positions, while also reducing repeated tool outputs and errors on BrowseComp and Terminal‑Bench 2.1.
arXiv:2610.34180v1 Announce Type: cross Abstract: Long-sequence models must retain earlier information. Full attention keeps all past tokens but grows expensive, and recurrent-memory models compress...
arXiv:2608. 12435v1 Announce Type: new Abstract: Transformers owe much of their strong long-context retrieval capability to a token-level memory that grows with context length.
arXiv:2607. 01523v1 Announce Type: cross Abstract: Recurrent memory agents extend LLMs to arbitrarily long contexts by iteratively consolidating input into a fixed-size memory window.
arXiv:2606. 28876v1 Announce Type: cross Abstract: Long-context language models often conflate two different goals: compressing history into an efficient state, and maintaining reliable long-term memory.
arXiv:2606. 29916v1 Announce Type: cross Abstract: Long-running language agents need mechanisms for deciding which experiences should persist after the working context is gone.
arXiv:2609.39661v1 Announce Type: new Abstract: Self-attention gives LLMs fine-grained, query-dependent access to context, but dense token interactions incur quadratic prefill cost and a key--value c...
arXiv:2601.12075v2 Announce Type: replace Abstract: Language models used in retrieval-augmented settings must arbitrate between parametric knowledge stored in their weights and contextual information...
The paper introduces ATLAS, a method that uses retained-domain activation atlases to guide task-specific fine‑tuning of language models. By providing local reference centers and directional filters, ATLAS learns a low‑rank residual that adapts to new tasks while preserving existing behavior. Experiments on Qwen3‑8B and other backbones show lower retained‑output KL divergence, fewer rewritten answers, and more stable responses compared to seven baselines.
arXiv:2610.09835v1 Announce Type: new Abstract: Continual pre-training and fine-tuning in Large Language Models (LLMs) inevitably induce catastrophic forgetting, typically mitigated by replay using o...