arXiv AI
Scaling Attention Head Analysis via Gradient-Based Attribution in Context-Aware Machine Translation
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
Hugging Face Trending Papers
Jul 20
Token-Level Off-Policy Learning for Faithful Generation Under Distribution Shift
We propose Token-Level Off-Policy Labeling (TOPL), an off-policy training paradigm that reframes post-training as a token-level correctness prediction task. Our key intuition is that by training the model to distinguish good and bad tokens in a response, we naturally guide the model towards generating good tokens, while avoiding the pitfalls that come with directly training the model to generate off-policy tokens.
arXiv Machine Learning
Jul 21
Token-Level Off-Policy Learning for Faithful Generation Under Distribution Shift
arXiv:2607. 17524v1 Announce Type: cross Abstract: We propose Token-Level Off-Policy Labeling (TOPL), an off-policy training paradigm that reframes post-training as a token-level correctness prediction task.
arXiv AI
Jun 9
Contribution Weights: A Geometrical Analysis of Self-Attention Transformers
arXiv:2606. 07604v1 Announce Type: cross Abstract: Analyzing attention weights has become a standard approach for interpreting the information flow of Large Language Models (LLMs).
arXiv Computation and Language
Sep 2
Separating Syntax from Language: A Mechanistic Account of Translation in Multilingual LLMs
arXiv:2609.01356v1 Announce Type: new Abstract: Multilingual large language models (mLLMs) achieve strong performance in machine translation, yet our understanding of the mechanisms by which they tra...
arXiv AI
Jun 8
RePo: Language Models with Context Re-Positioning
arXiv:2512. 14391v3 Announce Type: replace-cross Abstract: In-context learning is fundamental to modern Large Language Models (LLMs); however, prevailing architectures impose a rigid and fixed contextual structure by assigning linear or constant positional indices.
arXiv Machine Learning
Jun 4
Fast & Faithful Function Vectors
arXiv:2606. 05079v1 Announce Type: cross Abstract: Function vectors (FVs) are task representations elicited during in-context learning that can be used to steer Large Language Models (LLMs).