arXiv AI
Write What Matters: Disentangling Token Selection from Slot Allocation in Recurrent Memory
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Jun 30
Memory-Managed Long-Context Attention: A Preliminary Study of Editable Request-Local Memory
arXiv:2606. 28876v1 Announce Type: cross Abstract: Long-context language models often conflate two different goals: compressing history into an efficient state, and maintaining reliable long-term memory.
arXiv AI
Jul 29
Raven: High-Recall Sequence Modeling with Sparse Memory Routing
arXiv:2607. 25357v1 Announce Type: cross Abstract: Long-context recall in linear-time sequence models highlights a tradeoff in how they write to memory.
arXiv Machine Learning
Jul 13
Memory-Managed Long-Context Attention: Bounded Editable Memory with a Hard Lifecycle and Calibrated Sparse Fallback
arXiv:2606. 28876v2 Announce Type: replace-cross Abstract: We study memory-managed long-context attention: explicit bounded memory with a learned query-independent writer, lifecycle control, query-aware reading, calibrated sparse fallback, and frozen-LLM generation from raw evidence.
arXiv Machine Learning
Aug 14
MARCH: Scaling Recurrent Memory with Content-Routed State Anchors
arXiv:2608. 12435v1 Announce Type: new Abstract: Transformers owe much of their strong long-context retrieval capability to a token-level memory that grows with context length.
arXiv Machine Learning
Oct 7
RAM-Net: Linear-Time Sequence Modeling with Sparsely Addressable State
RAM‑Net introduces a linear‑time sequence model that replaces dense access to a shared recurrent state with sparse, address‑based access. The model arranges the state as a fixed‑size array of independent slots and uses an Address Decoder to map each token’s key or query to a small subset of slots, thereby reducing inter‑token interference. Empirical results show that RAM‑Net outperforms strong recurrent baselines on fine‑grained long‑range retrieval, achieves the lowest perplexity in commonsense reasoning, and accesses far fewer state elements per step than competing models.
arXiv Computation and Language
Oct 1
The Evolution of Attention in Large Language Models: Mechanisms, Trade-offs, and Emerging Trends
arXiv:2609.39661v1 Announce Type: new Abstract: Self-attention gives LLMs fine-grained, query-dependent access to context, but dense token interactions incur quadratic prefill cost and a key--value c...