arXiv Machine Learning
Jun 9
Still: Amortized KV Cache Compaction in a Single Forward Pass
arXiv:2606. 07878v1 Announce Type: new Abstract: The KV cache is the memory bottleneck of long-horizon language model deployment.
The Optimization Landscape of Learning Compacted Context Models
The paper investigates the challenges of optimizing compacted context models, particularly the KV cache, in continual learning scenarios. It identifies the optimization landscape as brittle and flat, and proposes a simplified Perceiver-based architecture that matches or surpasses full Perceiver transformers in continuous context compaction. Experiments on MCQ tasks in Finance, Legal, Gutenberg, and Code demonstrate the effectiveness of this approach.
Related stories
arXiv Machine Learning
Jun 25
Lifelong In-Context Learning with Transformers Requires Parametric Forms of Attention
arXiv:2606. 25342v1 Announce Type: new Abstract: Lifelong continual learning remains an obstacle on the path to human-like intelligence.
Hugging Face Trending Papers
Jun 8
End-to-End Context Compression at Scale
Long-context language model inference is bottlenecked by memory, as the KV cache grows with context length. Recent techniques to compress the KV cache fall short: they either degrade model quality substantially or require considerable time and compute to compress a single long prompt.
Hugging Face Trending Papers
Jun 24
Lifelong In-Context Learning with Transformers Requires Parametric Forms of Attention
Lifelong continual learning remains an obstacle on the path to human-like intelligence. Modern transformers show sparks of intelligence with in-context learning.
arXiv AI
Jun 9
End-to-End Context Compression at Scale
arXiv:2606. 09659v1 Announce Type: cross Abstract: Long-context language model inference is bottlenecked by memory, as the KV cache grows with context length.
Hugging Face Trending Papers
Aug 20
Learning how to Forget: Fine-tuning for Long-Context Sparse Attention
A lot of prior work addressed key-value (KV) cache selection and compression by sparse attention to enable long-context inference for transformer language models without excessive hardware budgets. We provide a new method for fine-tuning models with sparse attention.
arXiv Machine Learning
Jul 10
What to Keep, What to Forget: A Rate--Distortion View of Memory Compaction in LLMs and Agents
arXiv:2607. 08032v1 Announce Type: new Abstract: Large language models, and the agents built on them, spend an ever-growing share of their compute and memory on remembering: caching attention keys and values, carrying long prompts, maintaining recurrent state, and storing what happened in previous turns and sessions.
arXiv Computation and Language
Aug 21
Learning how to Forget: Fine-tuning for Long-Context Sparse Attention
arXiv:2608. 19920v1 Announce Type: new Abstract: A lot of prior work addressed key-value (KV) cache selection and compression by sparse attention to enable long-context inference for transformer language models without excessive hardware budgets.
arXiv AI
Jun 17
Position: Modular Memory is the Key to Continual Learning Agents
arXiv:2603. 01761v2 Announce Type: replace-cross Abstract: Foundation models have transformed machine learning through large-scale pretraining and increased test-time compute.
arXiv AI
Aug 12
Dynamic Context Adapters: Efficiently Infusing History into Vision-and-Language Models
arXiv:2608. 10525v1 Announce Type: cross Abstract: Historical context integration presents a fundamental challenge for Vision-Language Models (VLMs) in sequential decision-making tasks.
arXiv Machine Learning
Sep 10
Conditioned Initialization for Attention
arXiv:2609.07086v1 Announce Type: new Abstract: Transformers are a dominant architecture in modern machine learning, powering applications across vision, language, and beyond. At the core of their su...
arXiv Machine Learning
Jun 29
Prism Transformer: Progressive Head Schedules for Hierarchical Attention Processing
arXiv:2606. 27449v1 Announce Type: new Abstract: Multi-head attention conventionally partitions the hidden dimension equally across all heads at every layer, enforcing an identical representational subspace dimension (dh = dmodel/h) throughout the models depth.