arXiv AI June 11, 2026 By Yuqi Zhang, Di Zhang

The Structural Attention Tax: How Retrieval Format Hijacks In-Context Learning Independent of Content

arXiv:2606. 11198v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) systems inject external knowledge to improve LLM outputs, yet the format of injected content -- distinct from its semantic relevance -- can independently distort the model's attention distribution.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv AI

Jul 24

Attention Degradation, Function Token Anchoring, and the Limits of Attention-Based Intervention in Large Language Models

arXiv:2607. 20524v1 Announce Type: new Abstract: Mean cross-positional attention degradation is widely reported in transformer interpretability, yet whether it causally limits contextual retrieval remains untested.

By Sagar Dangal, Manoj Shakya

llms safety

arXiv AI

Jul 28

Through the Bottleneck: How Multi-head Latent Attention Separates Content from Position in Language Models

arXiv:2607. 23054v1 Announce Type: cross Abstract: Multi-head Latent Attention (MLA), introduced in DeepSeek-V2, compresses key-value pairs through a shared low-rank bottleneck (cKV), achieving 81% KV-cache reduction during inference.

By Dhruvil S, Fenil Sojitra, Ravirajsinh Chauhan

llms fine-tuning safety

arXiv AI

Aug 5

Distractor-Aware Truncation: Disentangling Context-Length Effects from Signal Loss in Long-Context LLM Benchmarks

arXiv:2608. 03297v1 Announce Type: new Abstract: A standard claim in the literature on retrieval-augmented and memory-augmented language models is that shorter context is better when the relevant information is preserved.

By Mohsen Arjmandi

llms rag benchmarks

arXiv AI

Jun 2

On the Limits of LLM Adaptability: Impact of Model-Internalized Priors on Annotation Task Performance

arXiv:2606. 00467v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly used for zero-shot annotation and LLM-as-a-judge tasks, yet their reliability hinges on how model-internalized priors interact with user-provided instructions.

By Etienne Casanova, Rafal Kocielnik, R. Michael Alvarez

llms rag safety

arXiv AI

Jul 7

Hierarchical Sparse Attention Done Right: Toward Infinite Context Modeling

arXiv:2607. 02980v1 Announce Type: cross Abstract: Scaling modern large language models (LLMs) to long contexts is limited by the quadratic computation cost, and poor length extrapolation of dense attention.

By Xiang Hu, Xinyu Wei, Hao Gu, Minshen Zhang, Tian Liang, Huayang Li, Lei Zhu, Yan Wang, Sirui Han, Yushi Bai, Kewei Tu, Haitao Mi, Leo Liang

llms

arXiv AI

Aug 12

Attention-Path Fragility as an Uncertainty Signal in Large Language Models

arXiv:2608. 11138v1 Announce Type: cross Abstract: We propose that a model's uncertainty about a token is reflected not only in the breadth of its output distribution but also in whether a confident prediction is \emph{fragile} under perturbation of its attention pathways.

By Minsoo Kim, Sungyoung Ji, Kisung Moon, Ilyong Yoon

llms benchmarks

Related stories

Attention Degradation, Function Token Anchoring, and the Limits of Attention-Based Intervention in Large Language Models

Through the Bottleneck: How Multi-head Latent Attention Separates Content from Position in Language Models

Distractor-Aware Truncation: Disentangling Context-Length Effects from Signal Loss in Long-Context LLM Benchmarks

On the Limits of LLM Adaptability: Impact of Model-Internalized Priors on Annotation Task Performance

Hierarchical Sparse Attention Done Right: Toward Infinite Context Modeling

Attention-Path Fragility as an Uncertainty Signal in Large Language Models