arXiv Machine Learning July 21, 2026 By Zhihua Liang

The Geometry of Semantic Space: A Continuous Geometric Framework for the Transformer Architecture

Read the original on arXiv Machine Learning →

arXiv:2607. 17146v1 Announce Type: cross Abstract: We present a continuous geometric framework that models the discrete algebraic operations of the Transformer architecture as an integro-differential equation (IDE) on a semantic fiber bundle $\calE = \calM \times \R^d$.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jul 7

Statistically Meaningful Geometry (SMG) Beyond the Euclidean Paradigm, with Application to Generative AI

arXiv:2607. 03329v1 Announce Type: new Abstract: Conventional uniform convergence bounds and empirical risk minimization break down in massive over-parameterized models, such as large language transformers and biological sequence networks.

By Bing Cheng, Yi-Shuai Niu, Howell Tong, Shing-Tung Yau

llms fine-tuning safety

arXiv AI

Jun 11

Unifying Learning Dynamics and Generalization in Transformers Scaling Law

arXiv:2512. 22088v3 Announce Type: replace-cross Abstract: The scaling law, a cornerstone of Large Language Model (LLM) development, predicts improvements in model performance with increasing computational resources.

By Chiwun Yang

llms

arXiv Machine Learning

Jun 2

Language Modeling with Hyperspherical Flows

arXiv:2605. 11125v3 Announce Type: replace Abstract: Discrete Diffusion Language Models progressed rapidly as an alternative to autoregressive (AR) models, motivated by their parallel generation abilities.

By Justin Deschenaux, Caglar Gulcehre

llms rag diffusion

arXiv Machine Learning

Jul 14

From Self-Attention to Connection Laplacian: A Unified Operator View of Transformers

arXiv:2607. 10677v1 Announce Type: new Abstract: Self-attention is a ubiquitous primitive in modern sequence models, yet its operator-level geometry is only partially understood.

By Binbin Lin, Wei Chen, Yalun Li, Wenxiao Wang, Jieping Ye, Xiaofei He

llms

arXiv Machine Learning

Jun 9

Trajectory Geometry of Transformer Representations Across Layers

arXiv:2606. 09287v1 Announce Type: new Abstract: Understanding how transformer representations evolve across layers, not merely what they encode, remains an open problem in mechanistic interpretability.

By Vishal Pandey, Gopal Singh

llms rag safety

arXiv AI

Jul 21

Phantom Transitions in Language Model Fine-Tuning: A Density-Matrix Analysis

arXiv:2606. 07559v2 Announce Type: replace-cross Abstract: Fine-tuning a language model often fails silently when its correct completion must outrank a near-synonym competitor.

By Vaibhav Prakash, Jayasri Dontabhaktuni

llms rag fine-tuning

Related stories

Statistically Meaningful Geometry (SMG) Beyond the Euclidean Paradigm, with Application to Generative AI

Unifying Learning Dynamics and Generalization in Transformers Scaling Law

Language Modeling with Hyperspherical Flows

From Self-Attention to Connection Laplacian: A Unified Operator View of Transformers

Trajectory Geometry of Transformer Representations Across Layers

Phantom Transitions in Language Model Fine-Tuning: A Density-Matrix Analysis