arXiv Computation and Language October 1, 2026 By Eduard Tulchinskii, Kristian Kuznetsov, Laida Kushnareva, Anastasia Voznyuk, Andrei Andriiainen, Irina Piontkovskaya, Evgeny Burnaev, Serguei Barannikov

Listening to the Wise Few: Query-Key Alignment Unlocks Latent Correct Answers in Large Language Models

Read the original on arXiv Computation and Language →

The Flow has not summarised this story yet — read it at arXiv Computation and Language.

arXiv Computation and Language

Sep 15

Identifying Crucial Attention Heads for Multilingual Language Models: Retrieval and Retrieval-Transition Heads

arXiv:2602.22453v4 Announce Type: replace Abstract: Retrieval heads, a subset of attention heads in Transformers, were studied in English, showing its crucial role in retrieving information from the...

By Shaswat Patel, Vishvesh Trivedi, Yue Han, Yihuai Hong, Eunsol Choi

llms benchmarks

arXiv Computation and Language

1d ago

RoPE at the End of Its Rope? Theory, Diagnosis, and Mitigation of Long-Context Failures

arXiv:2609.39929v1 Announce Type: cross Abstract: Long-context failures of RoPE-based language models can arise from RoPE's intrinsic tradeoff between maintaining stable token preferences and disting...

By Yuyang Wu, Yufeng Du, Hao Peng

llms benchmarks

arXiv AI

Aug 18

Measuring Reward Hacking and Reasoning-Answer Decoupling Under Position-Confounded Optimization

arXiv:2608. 15445v1 Announce Type: new Abstract: When a reward is correct on every training example yet consistent with more than one goal, a model can acquire an unintended one, a failure known as goal misgeneralization.

By Suyash Maniyar, Armaan Sandhu, Abhishek Mishra

llms benchmarks

arXiv AI

Jul 2

Logit-Contribution Scoring Identifies Non-Literal Retrieval Heads

arXiv:2607. 01002v1 Announce Type: cross Abstract: In long-context use, large language models frequently synthesize answers from the meaning of a relevant context span rather than literally copy-pasting them.

By Aryo Pradipta Gema, Beatrice Alex, Pasquale Minervini

llms benchmarks

arXiv AI

Aug 7

Beyond Sequence Order: Syntax-Informed Positional Embeddings for Transformers

arXiv:2608. 06111v1 Announce Type: cross Abstract: Positional embeddings (PE) in Transformers encode token distance and order but are largely agnostic to \textit{syntactic structure}.

By Haris Riaz, Hyungji Kim, Mihai Surdeanu

llms rag benchmarks

arXiv Machine Learning

Jun 2

Test-Time Compute for Frozen Embedding Models through Agentic Program Search

arXiv:2605. 11374v5 Announce Type: replace Abstract: Test-time compute is widely believed to benefit only large reasoning models, leaving small models with nothing to gain.

By Han Xiao

llms rag agents

One email a morning, machine-written

Related stories

Identifying Crucial Attention Heads for Multilingual Language Models: Retrieval and Retrieval-Transition Heads

RoPE at the End of Its Rope? Theory, Diagnosis, and Mitigation of Long-Context Failures

Measuring Reward Hacking and Reasoning-Answer Decoupling Under Position-Confounded Optimization

Logit-Contribution Scoring Identifies Non-Literal Retrieval Heads

Beyond Sequence Order: Syntax-Informed Positional Embeddings for Transformers

Test-Time Compute for Frozen Embedding Models through Agentic Program Search