arXiv Machine Learning
Muir: Linear Attention with Sensitivity-Allocated Gating and Matched-Filter Readout
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Sep 1
DASC: Decay-Aware State Compression for Hybrid Linear-Attention Serving
arXiv:2608.30386v1 Announce Type: cross Abstract: Hybrid linear-attention architectures have recently scaled to large open-weight models, offering quality competitive with full attention while substa...
arXiv AI
Jul 21
Kernelized Linear Attention: Breaking the Capacity Wall with Symmetric Cones
arXiv:2607. 17419v1 Announce Type: cross Abstract: Linear attention promises constant-time recurrent inference but degrades sharply on associative recall.
arXiv Machine Learning
Jul 15
Forgetful Attention: A Trainable Support-Vector Memory with Certified Selection and Exact Unlearning
arXiv:2607. 12204v1 Announce Type: new Abstract: Attention can be viewed as an online learner over context, yet existing test-time memories cannot certify that dropping a token leaves outputs unchanged or delete its influence outright.
arXiv AI
Jun 26
CARVE: Content-Aware Recurrent with Value Efficiency for Chunk-Parallel Linear Attention
arXiv:2606. 27229v1 Announce Type: cross Abstract: Recurrent models must forget in order to remember, yet the state of the art decides what to erase without consulting what is stored -- the gate sees only the arriving token, not the memory it is about to modify.
arXiv Machine Learning
Jun 4
LimiX-2M: Mitigating Low-Rank Collapse and Attention Bottlenecks in Tabular Foundation Models
arXiv:2606. 04485v1 Announce Type: new Abstract: Tabular foundation models (TFMs) increasingly rival tree ensembles, but their performance is often compute-inefficient: with standard affine scalar tokenization, each feature injects value variation through an essentially one-dimensional channel, and feature IDs/positional signals cannot increase within-feature value degrees of freedom, yielding weak early-layer value sensitivity and redundant hidden states.
arXiv AI
Jul 10
Linear Attention Architectures: Mechanisms, Trade-offs, and Cross-Layer Routing
arXiv:2607. 07953v1 Announce Type: cross Abstract: Self-attention lets each token retrieve information from the full context, but its quadratic cost in sequence length limits training and inference at long context.