arXiv Machine Learning June 9, 2026 By Vishal Pandey, Gopal Singh

Trajectory Geometry of Transformer Representations Across Layers

Read the original on arXiv Machine Learning →

arXiv:2606. 09287v1 Announce Type: new Abstract: Understanding how transformer representations evolve across layers, not merely what they encode, remains an open problem in mechanistic interpretability.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Jun 3

Decomposing how prompting steers behavior

arXiv:2606. 03093v1 Announce Type: new Abstract: Prompting steers large language models (LLMs) and vision-language models (VLMs) without weight updates, but it remains unclear how instruction changes reshape internal representations to produce behavior.

By Fan L. Cheng, Nikolaus Kriegeskorte

llms multimodal

arXiv Machine Learning

Aug 4

ChaosProbe: A Neurochaotic Lens on Frozen Transformer Input-Embedding Spaces

arXiv:2608. 01968v1 Announce Type: new Abstract: Transformer models are most often understood through what they do: their benchmark performance, generation quality, or behavior on downstream tasks.

By Kunal Kumar Pant, Nithin Nagaraj

llms rag benchmarks

arXiv Machine Learning

Sep 1

Tracing distinguishability through transformer processing with stochastic LayerNorm

arXiv:2608.30720v1 Announce Type: new Abstract: Representational similarity is foundational to analyses of deep networks, yet distances between point-valued representations are not intrinsically tied...

By Kieran Murphy

llms fine-tuning efficiency safety

arXiv Machine Learning

Sep 22

Comparing Latent Concept Formation in State Space Models and Transformers via Sparse Autoencoders

The paper compares latent representations in Selective State Space Models (SSMs) like Mamba and Transformers such as Pythia using Sparse Autoencoders. Across a 10‑million token corpus, 99.98% of Mamba features align closely with Pythia’s, supporting the Universality Hypothesis that core semantic representations are similar across architectures. A tiny 0.02% of features diverge, with Mamba’s recurrent bottleneck causing it to compress syntactic anomalies into polysemantic neurons, whereas Pythia’s attention can isolate distinct formatting edge‑cases.

By Rithin Nagaraj, Rupa Laalasa Oruganti, Prerna Subhashchandra Kunder, Ashwini M Joshi

llms safety

arXiv AI

Jul 7

Emergent Semantic Representations in World Models through Physical Interaction without Linguistic Supervision

arXiv:2605. 28865v2 Announce Type: replace-cross Abstract: What does a world model learn from physical exploration, without any linguistic supervision?

By Jiayi Fang

agents diffusion robotics safety

arXiv Machine Learning

Jul 3

Transformer Geometry Observatory TGO-II: Representational Similarity Observatory

arXiv:2607. 02386v1 Announce Type: cross Abstract: While Vision Transformers have achieved remarkable success across computer vision and language applications, the geometric evolution of their internal representations throughout training remains insufficiently understood.

By Kaustubh Kapil, Kishor P. Upla

llms computer-vision safety

One email a morning, machine-written

Related stories

Decomposing how prompting steers behavior

ChaosProbe: A Neurochaotic Lens on Frozen Transformer Input-Embedding Spaces

Tracing distinguishability through transformer processing with stochastic LayerNorm

Comparing Latent Concept Formation in State Space Models and Transformers via Sparse Autoencoders

Emergent Semantic Representations in World Models through Physical Interaction without Linguistic Supervision

Transformer Geometry Observatory TGO-II: Representational Similarity Observatory