arXiv AI July 24, 2026 By Swamynathan V P

SR-TTT Does Not Learn Retrieval: A Correction and Mechanistic Post-Mortem of Surprisal-Aware Residual Test-Time Training

arXiv:2603. 06642v2 Announce Type: replace-cross Abstract: Test-Time Training (TTT) language models replace the KV-cache with fast weights updated during inference, achieving O(1) memory but suffering catastrophic failure on exact-recall tasks.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

llms

arXiv Machine Learning

Jul 13

Memory-Managed Long-Context Attention: Bounded Editable Memory with a Hard Lifecycle and Calibrated Sparse Fallback

arXiv:2606. 28876v2 Announce Type: replace-cross Abstract: We study memory-managed long-context attention: explicit bounded memory with a learned query-independent writer, lifecycle control, query-aware reading, calibrated sparse fallback, and frozen-LLM generation from raw evidence.

By Junyi Zou, Avrova Donz

llms

arXiv Machine Learning

Jun 30

Memory-Managed Long-Context Attention: A Preliminary Study of Editable Request-Local Memory

arXiv:2606. 28876v1 Announce Type: cross Abstract: Long-context language models often conflate two different goals: compressing history into an efficient state, and maintaining reliable long-term memory.

By Junyi Zou, Avrova Donz

llms benchmarks

arXiv Machine Learning

Jun 10

Parallel Causal Associative Fields: Gated Sparse Memory for Long-Context Language Modeling

arXiv:2606. 10435v1 Announce Type: new Abstract: Transformers achieve strong language modeling performance by providing direct token-to-token communication paths, but causal self-attention scales quadratically with context length.

By Muhammad Ahmed

llms

arXiv Machine Learning

Aug 7

QEvict: Recoverable Quantized KV Eviction for Attention-Drift-Robust Long-Context Decoding

arXiv:2608. 05326v1 Announce Type: new Abstract: Autoregressive large language model inference is increasingly constrained by the memory footprint of the Key-Value (KV) cache.

By Ayushman Garg, Akshita Gupta, Shaswata Bhattacharya, Abhishek Gupta, Sandeep Kumar, Manoj Kumar

llms efficiency benchmarks

arXiv AI

Jul 3

InduceKV: Fixed-Footprint Continual Adaptation of Multimodal LLMs via Inducing KV Memories

arXiv:2607. 02010v1 Announce Type: new Abstract: Multimodal large language models must adapt to evolving tasks and domains, yet continual improvement under bounded deployment footprint remains difficult because repeated parameter updates or growing replay stores can accumulate adaptation state over time.

By Qianyu Chen, Ziteng Feng, Canran Xiao, Runxuan Tang

llms fine-tuning multimodal

arXiv Machine Learning

Jul 15

Forgetful Attention: A Trainable Support-Vector Memory with Certified Selection and Exact Unlearning

arXiv:2607. 12204v1 Announce Type: new Abstract: Attention can be viewed as an online learner over context, yet existing test-time memories cannot certify that dropping a token leaves outputs unchanged or delete its influence outright.

By Vishwajith Ramesh

llms rag

Related stories

Memory-Managed Long-Context Attention: Bounded Editable Memory with a Hard Lifecycle and Calibrated Sparse Fallback

Memory-Managed Long-Context Attention: A Preliminary Study of Editable Request-Local Memory

Parallel Causal Associative Fields: Gated Sparse Memory for Long-Context Language Modeling

QEvict: Recoverable Quantized KV Eviction for Attention-Drift-Robust Long-Context Decoding

InduceKV: Fixed-Footprint Continual Adaptation of Multimodal LLMs via Inducing KV Memories

Forgetful Attention: A Trainable Support-Vector Memory with Certified Selection and Exact Unlearning