arXiv AI
From Relation-Specific Heads to Shared Interventions: Contextual Entrainment in Language Models
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jun 11
The Structural Attention Tax: How Retrieval Format Hijacks In-Context Learning Independent of Content
arXiv:2606. 11198v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) systems inject external knowledge to improve LLM outputs, yet the format of injected content -- distinct from its semantic relevance -- can independently distort the model's attention distribution.
arXiv Computation and Language
Sep 1
To Copy or Not to Copy: Copying Is Easier to Induce Than Recall
arXiv:2601.12075v2 Announce Type: replace Abstract: Language models used in retrieval-augmented settings must arbitrate between parametric knowledge stored in their weights and contextual information...
arXiv AI
Aug 13
Information Abundance Paradox: Long-Context Training Undermines Parametric Knowledge
arXiv:2608. 12218v1 Announce Type: cross Abstract: Large language models are increasingly trained and deployed with long contexts that span documents, code repositories, and interaction histories.
arXiv Machine Learning
Aug 11
Persistent Priors, Preserved Targets: A Stroop-Style Paradigm for Lexical Override
arXiv:2606. 07555v5 Announce Type: replace-cross Abstract: Local definitions can assign a familiar word a temporary meaning while its usual associations remain useful elsewhere.
arXiv Machine Learning
Jun 9
Priors Persist Through Suppression: A Stroop Paradigm for Lexical Override
arXiv:2606. 07555v1 Announce Type: cross Abstract: Glossaries, technical specifications, and system prompts routinely ask language models to use familiar words in unfamiliar ways.
Hugging Face Trending Papers
Jun 23
Sentence-Level Contextual Entrainment in Large Language Models
Contextual entrainment, which is a newly discovered phenomenon in large language models (LLMs), refers to the tendency of a model to assign higher probabilities to tokens that appear in its context. In this work, we extend this phenomenon from the token level to the sentence level by examining the per-token mean log-probability of a sentence instead of the probabilities of individual tokens.