arXiv Machine Learning August 31, 2026 By Mu Qiao

Attention as Conditioning: What Classical Learning Theory Predicts About Linear Transformers

Read the original on arXiv Machine Learning →

arXiv:2508. 08289v3 Announce Type: replace Abstract: Attention is widely understood as an associative memory, but that description alone does not predict how the memory will behave.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

llms

arXiv AI

Jul 3

A Hippocampus for Linear Attention: An Exact Memory for What the Recurrent State Forgets

arXiv:2607. 02303v1 Announce Type: new Abstract: Linear-attention and state-space language models compress the prefix into a fixed-size recurrent state, yielding O(1) memory at the cost of a lossy exact memory: when many key--value associations compete, earlier facts are overwritten and needle recall degrades.

By Wanyun Cui

llms efficiency

Hugging Face Trending Papers

Jul 2

A Hippocampus for Linear Attention: An Exact Memory for What the Recurrent State Forgets

Linear-attention and state-space language models compress the prefix into a fixed-size recurrent state, yielding O(1) memory at the cost of a lossy exact memory: when many key--value associations compete, earlier facts are overwritten and needle recall degrades. Inspired by Complementary Learning Systems, we give linear attention a hippocampal complement.

llms efficiency

arXiv Machine Learning

Jul 22

Relative Positions Generalize, Absolute Positions Memorize: An Implicit-Bias Account of Length Generalization in Attention

arXiv:2607. 18759v1 Announce Type: new Abstract: Transformers with relative positional encodings often extrapolate to sequences longer than those seen during training, whereas transformers with learned absolute encodings typically do not.

By Subham Singh, Ashutosh Mishra, Subha Raut

llms safety

arXiv Machine Learning

Sep 16

Anatomy of Associative Recall in Fixed-State Recurrences: A Matched-State Decomposition, an Interference Wall, and a Curriculum That Breaks It

arXiv:2609.16183v1 Announce Type: new Abstract: Fixed-state recurrences--linear attention and state-space models--are reported to lag behind attention on associative recall, but whole-architecture co...

By Julian Boesch, Andrew Wee

benchmarks safety

arXiv Machine Learning

Jun 18

Decomposing Prediction Mechanisms for In-Context Recall

arXiv:2507. 01414v2 Announce Type: replace Abstract: We introduce a new family of toy problems that combine features of linear-regression-style continuous in-context learning (ICL) with discrete associative recall.

By Sultan Daniels, Dylan Davis, Dhruv Gautam, Wentinn Liao, Gireeja Ranade, Anant Sahai

llms efficiency

arXiv Machine Learning

Oct 15

Memory by Design: Bayesian Layers for Sequence Modeling

arXiv:2605.31163v5 Announce Type: replace-cross Abstract: We introduce the \emph{design-model framework}: a way to derive efficient recurrent sequence maps from explicit assumptions about memory. A d...

By Matthew Dowling, Hyungju Jeon, Cristina Savin, Il Memming Park

benchmarks

One email a morning, machine-written

Related stories

A Hippocampus for Linear Attention: An Exact Memory for What the Recurrent State Forgets

A Hippocampus for Linear Attention: An Exact Memory for What the Recurrent State Forgets

Relative Positions Generalize, Absolute Positions Memorize: An Implicit-Bias Account of Length Generalization in Attention

Anatomy of Associative Recall in Fixed-State Recurrences: A Matched-State Decomposition, an Interference Wall, and a Curriculum That Breaks It

Decomposing Prediction Mechanisms for In-Context Recall

Memory by Design: Bayesian Layers for Sequence Modeling