arXiv Machine Learning
LoGo: Token-Level Dynamic Local-Global Attention
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Computation and Language
4d ago
SAS: Simple Attention Sparsification via End-to-End Optimization of Context Ranking
arXiv:2609.13141v1 Announce Type: new Abstract: Post-training attention sparsification reduces the quadratic cumulative attention cost of pretrained Transformers by selecting a small set of context u...
arXiv Machine Learning
Jul 7
Learning When to Attend: Conditional Memory Access for Long-Context LLMs
arXiv:2603. 17484v2 Announce Type: replace-cross Abstract: Language models struggle to generalize beyond pretraining context lengths, limiting long-horizon reasoning and retrieval.
arXiv AI
Jun 9
Full Attention Strikes Back: Transferring Full Attention into Sparse within Hundred Training Steps
arXiv:2605. 16928v2 Announce Type: replace-cross Abstract: Long-context inference in large language models is bottlenecked by the quadratic cost of full attention.
arXiv AI
Sep 3
Language Models Can Control Their Own Attention
The paper introduces Declarative Attention (DA), a protocol that lets language models explicitly declare which parts of their context to focus on during generation. By partitioning decoding into full-context, region-specific, and recent-output-only modes, the inference engine can skip large portions of the KV cache, dramatically reducing attended tokens. Experiments on 15 long-context tasks with off-the-shelf models show significant savings (52.0% and 31.1% reductions) with only modest accuracy drops that diminish as model size increases.
arXiv AI
Jul 9
TriRoute: Unified Learned Routing for Joint Adaptive Attention, Experts, and KV-Cache Allocation
arXiv:2607. 06601v1 Announce Type: cross Abstract: Conditional computation can decouple language model quality from per-token inference cost, yet leading techniques act on a single axis in isolation: Mixture-of-Experts (MoE) sparsifies the FFN, Mixture-of-Depths (MoD) skips whole transformer blocks, and KV-cache quantization compresses attention memory.
arXiv AI
Jun 18
Dual Dimensionality for Local and Global Attention
arXiv:2606. 18587v1 Announce Type: cross Abstract: Decoder-only Transformers compute attention over the KV cache of preceding tokens.