arXiv Machine Learning June 9, 2026 By Chenxiao Yang, Nathan Srebro, Zhiyuan Li

Tight Sample Complexity of Transformers

Read the original on arXiv Machine Learning →

arXiv:2606. 09731v1 Announce Type: new Abstract: We tightly characterize the VC dimension of depth-$L$ Transformers with a total of $W$ parameters, mapping an input sequence of length $T$ to a single output, establishing an upper bound of $O(L W \log (T W))$ and a nearly matching lower bound of $\Omega(L W \log (T W / L))$.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

llms

arXiv Machine Learning

Jul 14

From Expressivity to Sample Complexity: Narrow Teachers for Transformers via C-RASP

arXiv:2607. 11760v1 Announce Type: new Abstract: A theoretical understanding of Transformers is crucial to better understand the capacities and limitations of large language models (LLMs).

By Michael Rizvi-Martel, Satwik Bhattamishra, Guillaume Rabusseau, Michael Hahn

llms

arXiv Machine Learning

Aug 4

Understanding Rate-Distortion Performance in Distributed Transformer Inference

arXiv:2601. 22002v5 Announce Type: replace Abstract: Transformers achieve superior performance on many tasks, but impose heavy compute and memory requirements during inference.

By Anderson de Andrade, Alon Harell, Ivan V. Baji\'c

llms

arXiv Machine Learning

Sep 24

Even Sharper Bounds for Transductive Learning and Its Applications

arXiv:2609. 28459v1 Announce Type: new Abstract: We introduce Sharper Transductive Local Complexity (STLC), a localized complexity method for transductive learning under uniform sampling without replacement.

By Yingzhen Yang

arXiv Machine Learning

Sep 10

On Generalisation Error Bounds for Transformers

arXiv:2410. 11500v2 Announce Type: replace-cross Abstract: In this paper, we establish a collection of covering number bounds for linear function classes under various norm constraints on the inputs and matrices.

By Lan V. Truong

llms

arXiv Machine Learning

Sep 10

Length Generalization for Transformers via Compression

arXiv:2609.08851v1 Announce Type: new Abstract: Recent advancements in transformer length generalization theory enable us to reliably predict when a transformer can learn to solve a task. In particul...

By Georg Zetzsche, Hongjian Jiang, Andy Yang, Pascal Bergstr\"a{\ss}er, Marco S\"alzer, David Chiang, Anthony W. Lin

llms

arXiv Machine Learning

Aug 11

How Many Different Outputs Can a Transformer Generate?

arXiv:2605. 22223v2 Announce Type: replace Abstract: We study how we can leverage only a handful of characteristics of a transformer's architecture to closely predict the number of different sequences it can output, both qualitatively and quantitatively.

By Maxime Meyer, Mario Michelessa, Caroline Chaux, Vincent Y. F. Tan

llms

One email a morning, machine-written

Related stories

From Expressivity to Sample Complexity: Narrow Teachers for Transformers via C-RASP

Understanding Rate-Distortion Performance in Distributed Transformer Inference

Even Sharper Bounds for Transductive Learning and Its Applications

On Generalisation Error Bounds for Transformers

Length Generalization for Transformers via Compression

How Many Different Outputs Can a Transformer Generate?