arXiv Machine Learning June 2, 2026 By Lyumin Wu, Chenyang Zhang, Yuan Cao

Looped Transformers with Layer Normalization Provably Learn the Power Method

Read the original on arXiv Machine Learning →

arXiv:2606. 00605v1 Announce Type: new Abstract: Transformers have achieved remarkable success across a wide range of applications, and a growing body of work suggests that part of their strength comes from their ability to learn and execute algorithmic procedures.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv AI

Aug 10

Stability of Transformers under Layer Normalization

arXiv:2510. 09904v2 Announce Type: replace-cross Abstract: Despite their widespread use, training deep Transformers can be unstable.

By Kelvin Kan, Xingjian Li, Benjamin J. Zhang, Tuhin Sahai, Stanley Osher, Krishna Kumar, Markos A. Katsoulakis

llms

arXiv Machine Learning

Jun 5

SpanNorm: Reconciling Training Stability and Performance in Deep Transformers

arXiv:2601. 22580v2 Announce Type: replace-cross Abstract: The success of Large Language Models (LLMs) hinges on the stable training of deep Transformer architectures.

By Chao Wang, Bei Li, Jiaqi Zhang, Xinyu Liu, Yuchun Fan, Linkun Lyu, Xin Chen, Jingang Wang, Tong Xiao, Peng Pei, Xunliang Cai

llms

arXiv AI

Jul 20

In-context learning of closed form solution to simple linear regression task using transformer with linear self-attention

arXiv:2607. 15819v1 Announce Type: cross Abstract: In-context learning is a remarkable property of transformers and has recently received a lot of interest.

By Katsuyuki Hagiwara

llms

arXiv Machine Learning

Jul 14

AutoNorm: Understanding Adaptive Normalization in Transformers through Differentiable Gating

arXiv:2607. 10593v1 Announce Type: new Abstract: Normalization is a critical component for stabilizing Transformer training, yet the choice between static strategies such as Layer Normalization (LN) and adaptive alternatives remains largely task-dependent.

By Piyush Kaushik Bhattacharyya, Divyanshu Rai, Swastik Singh, Kumar Aakash, Ayush Ranjan, Krutika Verma

llms nlp benchmarks

arXiv Machine Learning

Jun 30

Generalization Analysis of Transformers in Distribution Regression

arXiv:2606. 29256v1 Announce Type: cross Abstract: In recent years, models based on the Transformer architecture have seen widespread applications and have become one of the core tools in the field of deep learning.

By Peilin Liu, Ding-Xuan Zhou

llms nlp fine-tuning

arXiv AI

Jun 11

Unifying Learning Dynamics and Generalization in Transformers Scaling Law

arXiv:2512. 22088v3 Announce Type: replace-cross Abstract: The scaling law, a cornerstone of Large Language Model (LLM) development, predicts improvements in model performance with increasing computational resources.

By Chiwun Yang

llms

Related stories

Stability of Transformers under Layer Normalization

SpanNorm: Reconciling Training Stability and Performance in Deep Transformers

In-context learning of closed form solution to simple linear regression task using transformer with linear self-attention

AutoNorm: Understanding Adaptive Normalization in Transformers through Differentiable Gating

Generalization Analysis of Transformers in Distribution Regression

Unifying Learning Dynamics and Generalization in Transformers Scaling Law