arXiv Machine Learning July 30, 2026 By Andy Gray

Transformers Can Learn Rules They've Never Seen: Proof of Computation Beyond Interpolation

Read the original on arXiv Machine Learning →

arXiv:2603. 17019v2 Announce Type: replace Abstract: A central question in the debate over large language models is whether transformers can learn rules they have never seen, or whether they can only interpolate: predict new cases from their similarity to training examples.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv AI

Jul 8

Transformers converge to invariant algorithmic cores

arXiv:2602. 22600v2 Announce Type: replace-cross Abstract: Training selects for behavior, not circuitry: many weight configurations can implement the same function.

By Joshua S. Schiffman

llms

arXiv Machine Learning

Aug 7

Transformers with RL or SFT Provably Learn Sparse Boolean Functions, But Differently

arXiv:2511. 17852v3 Announce Type: replace Abstract: Transformers can acquire Chain-of-Thought (CoT) capabilities to solve reasoning tasks via fine-tuning.

By Bochen Lyu, Yiyang Jia, Xiaohao Cai, Zhanxing Zhu

llms reinforcement-learning fine-tuning

arXiv Machine Learning

Aug 11

Barriers to Universal Reasoning With Transformers (And How to Overcome Them)

arXiv:2604. 25800v2 Announce Type: replace Abstract: Chain-of-Thought (CoT) has been shown to empirically improve Transformers' performance, and theoretically increase their expressivity to Turing completeness.

By Oliver Kraus, Yash Sarrof, Yuekun Yao, Alexander Koller, Michael Hahn

llms

arXiv AI

Jul 14

Invariant Learning Dynamics of Transformers in Inductive Reasoning Tasks

arXiv:2607. 11875v1 Announce Type: cross Abstract: We present a theoretical framework to explain the emergence of inductive reasoning abilities in Transformer language models.

By Tiberiu Musat, Tiago Pimentel, Nicholas Zucchet, Thomas Hofmann

llms

arXiv AI

Jun 2

Breaking the Reversal Curse in Autoregressive Language Models via Identity Bridge

arXiv:2602. 02470v2 Announce Type: replace Abstract: Autoregressive large language models (LLMs) have achieved remarkable success in many complex tasks, yet they can still fail in very simple logical reasoning such as the "reversal curse" -- when trained on forward knowledge data of the form "$A \rightarrow B$" (e.

By Xutao Ma, Yixiao Huang, Hanlin Zhu, Somayeh Sojoudi

llms safety

arXiv Machine Learning

Jul 1

Explicit Fuzzy Logic in the Feed-Forward Layer: Self-Forgetting Quantifiers Discover Legible Grammatical-Licensing Detectors

arXiv:2606. 31845v1 Announce Type: cross Abstract: A transformer's feed-forward (FFN) sublayer materializes the distinctions attention gathers, yet gives no account of what it computes.

By Mark Oskin

llms

Related stories

Transformers converge to invariant algorithmic cores

Transformers with RL or SFT Provably Learn Sparse Boolean Functions, But Differently

Barriers to Universal Reasoning With Transformers (And How to Overcome Them)

Invariant Learning Dynamics of Transformers in Inductive Reasoning Tasks

Breaking the Reversal Curse in Autoregressive Language Models via Identity Bridge

Explicit Fuzzy Logic in the Feed-Forward Layer: Self-Forgetting Quantifiers Discover Legible Grammatical-Licensing Detectors