arXiv Machine Learning August 11, 2026 By Maxime Meyer, Mario Michelessa, Caroline Chaux, Vincent Y. F. Tan

How Many Different Outputs Can a Transformer Generate?

Read the original on arXiv Machine Learning →

arXiv:2605. 22223v2 Announce Type: replace Abstract: We study how we can leverage only a handful of characteristics of a transformer's architecture to closely predict the number of different sequences it can output, both qualitatively and quantitatively.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

llms

arXiv Machine Learning

Sep 10

Length Generalization for Transformers via Compression

arXiv:2609.08851v1 Announce Type: new Abstract: Recent advancements in transformer length generalization theory enable us to reliably predict when a transformer can learn to solve a task. In particul...

By Georg Zetzsche, Hongjian Jiang, Andy Yang, Pascal Bergstr\"a{\ss}er, Marco S\"alzer, David Chiang, Anthony W. Lin

llms

arXiv AI

Jul 7

On the Ability of Transformers to Verify Plans

arXiv:2603. 19954v2 Announce Type: replace Abstract: Transformers have shown inconsistent success in AI planning tasks, and theoretical understanding of when generalization should be expected has been limited.

By Yash Sarrof, Yupei Du, Katharina Stein, Alexander Koller, Sylvie Thi\'ebaux, Michael Hahn

llms

arXiv AI

Aug 14

On the Expressive Power of Transformers

arXiv:2608. 12671v1 Announce Type: new Abstract: Multi-layer transformers form the critical component of essentially all large language models (LLMs) in use today.

By Phokion Kolaitis, Rik Sengupta

llms

arXiv Machine Learning

Aug 4

Understanding Rate-Distortion Performance in Distributed Transformer Inference

arXiv:2601. 22002v5 Announce Type: replace Abstract: Transformers achieve superior performance on many tasks, but impose heavy compute and memory requirements during inference.

By Anderson de Andrade, Alon Harell, Ivan V. Baji\'c

llms

arXiv Machine Learning

Sep 10

On Generalisation Error Bounds for Transformers

arXiv:2410. 11500v2 Announce Type: replace-cross Abstract: In this paper, we establish a collection of covering number bounds for linear function classes under various norm constraints on the inputs and matrices.

By Lan V. Truong

llms

arXiv Machine Learning

Jun 9

Understanding the Parameter Space Geometry of Transformers Encoding Boolean Functions

arXiv:2606. 08768v1 Announce Type: new Abstract: Transformers consistently fail to learn certain simple functions that are provably expressible with specific parameter settings.

By Blanka K\"over, Alexandra Butoi, Anej Svete, Michael Hahn, Ryan Cotterell

llms safety

One email a morning, machine-written

Related stories

Length Generalization for Transformers via Compression

On the Ability of Transformers to Verify Plans

On the Expressive Power of Transformers

Understanding Rate-Distortion Performance in Distributed Transformer Inference

On Generalisation Error Bounds for Transformers

Understanding the Parameter Space Geometry of Transformers Encoding Boolean Functions