arXiv Machine Learning
Emergent phases of superposition: from partial to full representation
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Jun 5
Generative Criticality in Large Language Model Temperature Scaling
arXiv:2606. 06238v1 Announce Type: new Abstract: We propose a statistical-field framework for text generated by large language models (LLMs), treating token embeddings as continuous spin variables on a one-dimensional chain.
arXiv AI
Jun 3
Representational Capacity: Geometric Limits on Feature Representation in Transformer Language Models
arXiv:2606. 02765v1 Announce Type: cross Abstract: Model dimension ($d_{model}$) is a fundamental hyperparameter in transformer language models, yet its role in setting the geometric limits of feature representation remains under-explored.
arXiv AI
Sep 15
Ventriloquist LLMs: Linear Alignment of Late-Stage Representations
arXiv:2603.18908v5 Announce Type: replace Abstract: Independently trained language models often learn compatible late-stage representations, despite differences in training objectives, architectures,...
arXiv Machine Learning
Jul 13
Neural Collapse Is Forbidden: Information Floors in Language Models
arXiv:2607. 09487v1 Announce Type: new Abstract: Within-class variance in language-model representations is commonly read as incomplete neural collapse.
arXiv Machine Learning
Sep 16
Attention Mean Fields Predict Average Representation Dynamics and Reveal Context-Specific Computation
The paper presents a mean‑field analysis of attention in language models, defining an average attention kernel that propagates representations layer by layer. When conditioned on a whole corpus, the kernel predicts the average evolution of representation geometry; when conditioned on a single context, it predicts the expected geometry for that context. The difference between actual attention and the mean‑field prediction—called the mean‑field deviation—captures context‑specific computation, revealing how models diverge from average behavior during training and in few‑shot tasks.
arXiv AI
Jun 11
Unstable Features, Reproducible Subspaces: Understanding Seed Dependence in Sparse Autoencoders
arXiv:2606. 12138v1 Announce Type: cross Abstract: Sparse autoencoders (SAEs) are widely used to interpret neural network representations, but their utility depends on whether the learned features are reproducible across training runs.