arXiv Machine Learning July 14, 2026 By Jorge A. Castillo, Marco Torres Y\'evenes, Juan Carlos Lanas

From Direction to Magnitude: How Multimodal Instruction-Tuning Reorganizes the Geometric Encoding of Identity-Specifying Prompts in Transformer Hidden States

Read the original on arXiv Machine Learning →

arXiv:2607. 09842v1 Announce Type: new Abstract: We investigate whether identity-specifying system prompts produce statistically distinguishable geometric fingerprints in the hidden-state trajectories of four open-weight transformer language models spanning four post-training regimes: no training (Gemma-4-E4B base), multimodal RLHF (Gemma-4-E4B-it), RL distillation (DeepSeek-R1-Distill-Qwen-7B), and SFT (Qwen2.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jun 2

Task Structure Reverses Layerwise State Encoding in Sequence Models

arXiv:2606. 00926v1 Announce Type: new Abstract: Mechanistic studies of sequence models often treat layerwise state encodings as architectural traits: recurrent models concentrate readable state, attention-based models distribute it.

By Yuhang Jiang

llms fine-tuning benchmarks

arXiv Machine Learning

5d ago

Geometric and Behavioral Stratification in Transformer Residual Streams

arXiv:2608. 12447v1 Announce Type: new Abstract: Trained transformer models develop privileged bases: coordinate axes whose statistics differ from the rest of the residual stream.

By Nelson Guda

llms

arXiv AI

Jul 8

Transformers converge to invariant algorithmic cores

arXiv:2602. 22600v2 Announce Type: replace-cross Abstract: Training selects for behavior, not circuitry: many weight configurations can implement the same function.

By Joshua S. Schiffman

llms

arXiv AI

Jun 2

MENTIS: What Belief Changes Under Alignment? Measuring Multi-Scale Latent Torsion in Language Models

arXiv:2606. 01060v1 Announce Type: cross Abstract: Preference alignment has substantially improved the observable behavior of large language models, yet it remains unclear what alignment changes internally.

By Partha Pratim Saha, Samarth Raina, Mayur Parvatikar, Amit Dhanda, Vinija Jain, Aman Chadha, Amitava Das

llms safety

arXiv Machine Learning

Jun 11

Kuramoto Attention: Synchronizing Self-Attention on the Torus

arXiv:2606. 11585v1 Announce Type: new Abstract: We introduce Kuramoto attention, a self-attention layer in which each hidden coordinate is an angle.

By Joshua Nunley

llms

arXiv Machine Learning

Jun 9

Trajectory Geometry of Transformer Representations Across Layers

arXiv:2606. 09287v1 Announce Type: new Abstract: Understanding how transformer representations evolve across layers, not merely what they encode, remains an open problem in mechanistic interpretability.

By Vishal Pandey, Gopal Singh

llms rag safety

Related stories

Task Structure Reverses Layerwise State Encoding in Sequence Models

Geometric and Behavioral Stratification in Transformer Residual Streams

Transformers converge to invariant algorithmic cores

MENTIS: What Belief Changes Under Alignment? Measuring Multi-Scale Latent Torsion in Language Models

Kuramoto Attention: Synchronizing Self-Attention on the Torus

Trajectory Geometry of Transformer Representations Across Layers