arXiv Machine Learning July 14, 2026 By Jorge A. Castillo, Marco Torres Y\'evenes, Juan Carlos Lanas

From Direction to Magnitude: How Multimodal Instruction-Tuning Reorganizes the Geometric Encoding of Identity-Specifying Prompts in Transformer Hidden States

Read the original on arXiv Machine Learning →

arXiv:2607. 09842v1 Announce Type: new Abstract: We investigate whether identity-specifying system prompts produce statistically distinguishable geometric fingerprints in the hidden-state trajectories of four open-weight transformer language models spanning four post-training regimes: no training (Gemma-4-E4B base), multimodal RLHF (Gemma-4-E4B-it), RL distillation (DeepSeek-R1-Distill-Qwen-7B), and SFT (Qwen2.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jun 2

Task Structure Reverses Layerwise State Encoding in Sequence Models

arXiv:2606. 00926v1 Announce Type: new Abstract: Mechanistic studies of sequence models often treat layerwise state encodings as architectural traits: recurrent models concentrate readable state, attention-based models distribute it.

By Yuhang Jiang

llms fine-tuning benchmarks

arXiv Machine Learning

Aug 14

Geometric and Behavioral Stratification in Transformer Residual Streams

arXiv:2608. 12447v1 Announce Type: new Abstract: Trained transformer models develop privileged bases: coordinate axes whose statistics differ from the rest of the residual stream.

By Nelson Guda

llms

arXiv AI

Jul 8

Transformers converge to invariant algorithmic cores

arXiv:2602. 22600v2 Announce Type: replace-cross Abstract: Training selects for behavior, not circuitry: many weight configurations can implement the same function.

By Joshua S. Schiffman

llms

arXiv Machine Learning

Aug 27

Spectral Allocation: Why Muon Outperforms Adam, and How to Improve Muon

The paper investigates why the orthogonal optimiser Muon outperforms Adam in large language model pretraining by analysing the spectral properties of Transformer loss landscapes. It finds that Muon’s momentum buffers exhibit an anisotropic spectral profile with a volatile head and a tolerant bulk, enabling larger effective step sizes. Building on this insight, the authors propose Spectral‑Aware Muon (SAMuon) and a lightweight variant, which adjust the bulk scaling while keeping the head unchanged, achieving 13–24 % fewer training tokens than Muon without extra FLOPs.

By Xiaodong Wu, Wenyi Yu, Chao Zhang, Philip Woodland

llms

arXiv AI

Jun 2

MENTIS: What Belief Changes Under Alignment? Measuring Multi-Scale Latent Torsion in Language Models

arXiv:2606. 01060v1 Announce Type: cross Abstract: Preference alignment has substantially improved the observable behavior of large language models, yet it remains unclear what alignment changes internally.

By Partha Pratim Saha, Samarth Raina, Mayur Parvatikar, Amit Dhanda, Vinija Jain, Aman Chadha, Amitava Das

llms safety

arXiv Machine Learning

Jun 11

Kuramoto Attention: Synchronizing Self-Attention on the Torus

arXiv:2606. 11585v1 Announce Type: new Abstract: We introduce Kuramoto attention, a self-attention layer in which each hidden coordinate is an angle.

By Joshua Nunley

llms

One email a morning, machine-written

Related stories

Task Structure Reverses Layerwise State Encoding in Sequence Models

Geometric and Behavioral Stratification in Transformer Residual Streams

Transformers converge to invariant algorithmic cores

Spectral Allocation: Why Muon Outperforms Adam, and How to Improve Muon

MENTIS: What Belief Changes Under Alignment? Measuring Multi-Scale Latent Torsion in Language Models

Kuramoto Attention: Synchronizing Self-Attention on the Torus