arXiv Machine Learning
Prediction Dynamics in Depth-Recurrent Language Models
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Jun 2
The Shape of Wisdom: Decision Trajectories in Language Models
arXiv:2606. 01202v1 Announce Type: new Abstract: Language models do not simply choose an answer at the output layer.
arXiv AI
Aug 7
Layer-wise Positional Bias in Short-Context Language Modeling
arXiv:2601. 04098v2 Announce Type: replace-cross Abstract: Transformer language models systematically prefer tokens at specific input positions regardless of semantic relevance---a phenomenon known as positional bias.
arXiv Machine Learning
Jul 7
How Much is Left? LLMs Linearly Encode Their Remaining Output Length
arXiv:2607. 05316v1 Announce Type: cross Abstract: Large language models generate one token at a time, yet their responses show remarkably consistent length structure: step-by-step solutions converge in predictable token counts, retrievals stop after a few sentences, retractions extend responses by measurable amounts.
arXiv AI
Sep 2
The Interlingua Hypothesis: LLMs Translate via a Latent Task-agnostic Feature Space
The paper proposes the interlingua hypothesis, suggesting that large language models translate by encoding a source sentence into a latent, task‑agnostic feature space and then decoding a target sentence from that space. Three lines of evidence support this: (1) BLEU variance across language pairs is largely explained by language‑specific competences without pair‑specific interactions; (2) many model components influence both monolingual and translation tasks; and (3) fine‑tuning on monolingual data recovers most translation gains seen with aligned documents. These findings converge to support the hypothesis and point toward new ways to understand and improve LLM translation.
arXiv AI
Sep 15
Ventriloquist LLMs: Linear Alignment of Late-Stage Representations
arXiv:2603.18908v5 Announce Type: replace Abstract: Independently trained language models often learn compatible late-stage representations, despite differences in training objectives, architectures,...
arXiv AI
Jul 28
Attention-Guided Layer Selection for Contrastive Decoding in Large Language Models
arXiv:2607. 23067v1 Announce Type: cross Abstract: Contrastive decoding methods such as DoLa improve the factuality of Large Language Models (LLMs) by contrasting the output distributions of mature and premature layers.