arXiv Machine Learning July 30, 2026 By Andy Gray

Transformers Can Learn Rules They've Never Seen: Proof of Computation Beyond Interpolation

Read the original on arXiv Machine Learning →

arXiv:2603. 17019v2 Announce Type: replace Abstract: A central question in the debate over large language models is whether transformers can learn rules they have never seen, or whether they can only interpolate: predict new cases from their similarity to training examples.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Jul 8

Transformers converge to invariant algorithmic cores

arXiv:2602. 22600v2 Announce Type: replace-cross Abstract: Training selects for behavior, not circuitry: many weight configurations can implement the same function.

By Joshua S. Schiffman

llms

arXiv Machine Learning

Aug 7

Transformers with RL or SFT Provably Learn Sparse Boolean Functions, But Differently

arXiv:2511. 17852v3 Announce Type: replace Abstract: Transformers can acquire Chain-of-Thought (CoT) capabilities to solve reasoning tasks via fine-tuning.

By Bochen Lyu, Yiyang Jia, Xiaohao Cai, Zhanxing Zhu

llms reinforcement-learning fine-tuning

arXiv Machine Learning

Sep 1

Universal Transformers for Circuit Computations: Perfect Length Generalization in Tiny Transformers

arXiv:2608.31067v1 Announce Type: new Abstract: Learning generalizable algorithmic computations remains a challenge for neural networks, as reflected in persistent failures on compositional and lengt...

By Takuya Ito, Ruchir Puri, Murray Campbell, Parikshit Ram

llms agents benchmarks

arXiv Machine Learning

Aug 11

Barriers to Universal Reasoning With Transformers (And How to Overcome Them)

arXiv:2604. 25800v2 Announce Type: replace Abstract: Chain-of-Thought (CoT) has been shown to empirically improve Transformers' performance, and theoretically increase their expressivity to Turing completeness.

By Oliver Kraus, Yash Sarrof, Yuekun Yao, Alexander Koller, Michael Hahn

llms

arXiv Computation and Language

4d ago

Pretraining Latent Information Feedback Transformers with Teacher Supervision

arXiv:2609.38149v1 Announce Type: new Abstract: Transformer language models (LMs) are feed-forward: deep-layer representations are never fed back to shallower layers, and the only pathway for informa...

By Dor Tirosh, Ido Amos, Mor Geva

llms

arXiv AI

Jul 14

Invariant Learning Dynamics of Transformers in Inductive Reasoning Tasks

arXiv:2607. 11875v1 Announce Type: cross Abstract: We present a theoretical framework to explain the emergence of inductive reasoning abilities in Transformer language models.

By Tiberiu Musat, Tiago Pimentel, Nicholas Zucchet, Thomas Hofmann

llms

One email a morning, machine-written

Related stories

Transformers converge to invariant algorithmic cores

Transformers with RL or SFT Provably Learn Sparse Boolean Functions, But Differently

Universal Transformers for Circuit Computations: Perfect Length Generalization in Tiny Transformers

Barriers to Universal Reasoning With Transformers (And How to Overcome Them)

Pretraining Latent Information Feedback Transformers with Teacher Supervision

Invariant Learning Dynamics of Transformers in Inductive Reasoning Tasks