arXiv AI
Jun 2
The Shape of Wisdom: Decision Trajectories in Language Models
arXiv:2606. 01202v1 Announce Type: new Abstract: Language models do not simply choose an answer at the output layer.
Prediction Dynamics in Depth-Recurrent Language Models
Related stories
arXiv AI
Aug 7
Layer-wise Positional Bias in Short-Context Language Modeling
arXiv:2601. 04098v2 Announce Type: replace-cross Abstract: Transformer language models systematically prefer tokens at specific input positions regardless of semantic relevance---a phenomenon known as positional bias.
arXiv Machine Learning
Jul 7
How Much is Left? LLMs Linearly Encode Their Remaining Output Length
arXiv:2607. 05316v1 Announce Type: cross Abstract: Large language models generate one token at a time, yet their responses show remarkably consistent length structure: step-by-step solutions converge in predictable token counts, retrievals stop after a few sentences, retractions extend responses by measurable amounts.
arXiv AI
Sep 2
The Interlingua Hypothesis: LLMs Translate via a Latent Task-agnostic Feature Space
The paper proposes the interlingua hypothesis, suggesting that large language models translate by encoding a source sentence into a latent, task‑agnostic feature space and then decoding a target sentence from that space. Three lines of evidence support this: (1) BLEU variance across language pairs is largely explained by language‑specific competences without pair‑specific interactions; (2) many model components influence both monolingual and translation tasks; and (3) fine‑tuning on monolingual data recovers most translation gains seen with aligned documents. These findings converge to support the hypothesis and point toward new ways to understand and improve LLM translation.
arXiv AI
Sep 15
Ventriloquist LLMs: Linear Alignment of Late-Stage Representations
arXiv:2603.18908v5 Announce Type: replace Abstract: Independently trained language models often learn compatible late-stage representations, despite differences in training objectives, architectures,...
arXiv AI
Jul 28
Attention-Guided Layer Selection for Contrastive Decoding in Large Language Models
arXiv:2607. 23067v1 Announce Type: cross Abstract: Contrastive decoding methods such as DoLa improve the factuality of Large Language Models (LLMs) by contrasting the output distributions of mature and premature layers.
arXiv AI
Jul 28
What Does Chain-of-Thought Contribute at Probe Time? Evidence for Local Co-Occurrence Activation
arXiv:2605. 26795v2 Announce Type: replace Abstract: Chain-of-thought (CoT) prompting enhances large language model performance, yet what drives these gains remains unclear.
arXiv AI
Jun 10
When Distance Distracts: Representation Distance Bias in BT-Loss for Reward Models
arXiv:2512. 06343v3 Announce Type: replace-cross Abstract: Reward models are central to Large Language Model (LLM) alignment within the framework of RLHF.
Hugging Face Trending Papers
Jul 6
How Much is Left? LLMs Linearly Encode Their Remaining Output Length
Large language models generate one token at a time, yet their responses show remarkably consistent length structure: step-by-step solutions converge in predictable token counts, retrievals stop after a few sentences, retractions extend responses by measurable amounts. We ask whether the model carries an internal estimate of how much response remains.
arXiv Machine Learning
Sep 24
ChronoSteer: Bridging Large Language Model and Time Series Foundation Model via Synthetic Cross-Modal Alignment Dataset
ChronoSteer is a decoupled agentic framework that bridges large language models and time series foundation models by learning cross‑modal alignment from synthetic paired supervision. It converts textual events into revision instructions that steer a frozen time‑series model, discretizes these instructions into a compact codebook to reduce semantic divergence, and then refines the predictions with a two‑stage training strategy. The authors also release a leakage‑controlled multimodal benchmark and report a 25.8% improvement in zero‑shot prediction accuracy over the unimodal backbone.
arXiv Machine Learning
1d ago
Persistent Depth Ordering amid Shifting Block-Bypass Responses in Language Model Pretraining
The study investigates how layer‑wise intervention responses in language models change over the course of pretraining, using single‑block identity bypass across multiple checkpoints and model‑domain combinations. It finds that while depth ordering of responses persists, their magnitudes shift, with nearby checkpoints showing stronger rank correspondence than distant ones and large changes occurring at positions that recur across samples and transfer across evaluation domains. Controlled experiments reveal that these longitudinal changes cannot be explained by a single downstream sensitivity and depend on perturbation strength and direction, indicating that layer sensitivity is structured but dynamic.
arXiv AI
Jun 2
MENTIS: What Belief Changes Under Alignment? Measuring Multi-Scale Latent Torsion in Language Models
arXiv:2606. 01060v1 Announce Type: cross Abstract: Preference alignment has substantially improved the observable behavior of large language models, yet it remains unclear what alignment changes internally.