← Back to all news
arXiv AI By Dongyu Gong

Language Models Share a Readout Bottleneck in Working Memory

Read the original on arXiv AI →

Large language models (LLMs) exhibit a human‑like working‑memory pattern on the N‑back task, with performance dropping as memory load increases and errors mainly on lure items. Two hypotheses—precision limits of softmax attention and interference from competing items—were tested across 15 pretrained LLMs and minimal transformers. Results show that interference, not precision, constrains LLMs’ working‑memory performance, as suppressing non‑target item identities improves sensitivity and larger models benefit more from reducing interference.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Sep 21

Decoupling Internal Representational Changes and Causal Importance in Fine-Tuned Large Language Models

Fine‑tuning reshapes internal representations of large language models, affecting attention patterns and layer‑wise activations. The study shows that components identified by EAP as important for task performance cluster in specific layers, yet these layers do not align with those undergoing the largest representational changes. Additionally, overlapping EAP components across different tasks do not guarantee cross‑task transfer and can even degrade performance when tasks differ in nature.

By Lingfang Li, Procheta Sen, Shubham Das, Danushka Bollegala
llmsfine-tuning
More like this →
arXiv Machine Learning
Sep 21

Abstention and Noise Filtering: Two Missing Primitives of Softmax Attention

The paper investigates how gating the value pathway in attention mechanisms provides two missing capabilities of softmax attention: abstention and noise filtering. Experiments on models ranging from 10M to 350M parameters show that abstention benefits smaller models while noise filtering becomes more advantageous as models scale, and that combining both primitives yields the best performance across all sizes. The authors also demonstrate that the gates effectively suppress interference and that each gate type has a distinct blind spot, all while adding negligible parameters and preserving compatibility with key‑value caching.

By Richard Zhe Wang
llms
More like this →
arXiv AI
Sep 3

Language Models Can Control Their Own Attention

The paper introduces Declarative Attention (DA), a protocol that lets language models explicitly declare which parts of their context to focus on during generation. By partitioning decoding into full-context, region-specific, and recent-output-only modes, the inference engine can skip large portions of the KV cache, dramatically reducing attended tokens. Experiments on 15 long-context tasks with off-the-shelf models show significant savings (52.0% and 31.1% reductions) with only modest accuracy drops that diminish as model size increases.

By Namgyu Ho, Huzama Ahmad, Woosung Koh, Se-Young Yun, Tal Schuster, Cicero Nogueira dos Santos
llmsefficiency
More like this →