arXiv Machine Learning August 10, 2026 By Moritz Br\"osamle, Stephan Eckstein

The Expressive Power of Low Precision Softmax Transformers with (Summarized) Chain-of-Thought

Read the original on arXiv Machine Learning →

arXiv:2605. 18079v2 Announce Type: replace Abstract: Existing expressivity results for transformers typically rely on hardmax attention, high precision, and other architectural modifications that disconnect them from the models used in practice.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Aug 11

Barriers to Universal Reasoning With Transformers (And How to Overcome Them)

arXiv:2604. 25800v2 Announce Type: replace Abstract: Chain-of-Thought (CoT) has been shown to empirically improve Transformers' performance, and theoretically increase their expressivity to Turing completeness.

By Oliver Kraus, Yash Sarrof, Yuekun Yao, Alexander Koller, Michael Hahn

llms

arXiv Machine Learning

Aug 11

Training-Free Universal Approximation by Prompting Random Transformers

arXiv:2608. 09558v1 Announce Type: new Abstract: How expressive is prompting a transformer?

By Alexander Hsu, Rongjie Lai

llms

arXiv AI

Jun 18

Explaining Attention with Program Synthesis

arXiv:2606. 19317v1 Announce Type: cross Abstract: A longstanding goal of research on interpretable deep learning is to replace opaque neural computations with human-meaningful symbolic descriptions.

By Amiri Hayes, Belinda Li, Jacob Andreas

llms nlp benchmarks

arXiv Machine Learning

Jul 21

Can Transformers Really Do It All? On the Compatibility of Inductive Biases Across Tasks

arXiv:2607. 17624v1 Announce Type: new Abstract: Transformers are remarkably versatile and their design is largely consistent across a variety of applications.

By Damien Teney, Liangze Jiang, Hemanth Saratchandran, Simon Lucey

llms

arXiv AI

Jun 30

Emergence of Minimal Circuits for Indirect Object Identification in Attention-Only Transformers

arXiv:2510. 25013v2 Announce Type: replace-cross Abstract: Mechanistic interpretability aims to reverse-engineer large language models (LLMs) into human-understandable computational circuits.

By Rabin Adhikari

llms rag benchmarks safety

arXiv AI

Jun 19

Efficiently Representing Algorithms With Chain-of-Thought Transformers

arXiv:2606. 19697v1 Announce Type: cross Abstract: The increasing popularity of \emph{reasoning} models -- language models that output a series of reasoning or thought tokens before producing an answer -- is justified, in part, by theoretical results showing that chain-of-thought (CoT) transformers can simulate Turing machines, and thus perform arbitrary computation.

By Yanhong Li, Anej Svete, Ashish Sabharwal, William Merrill

llms

Related stories

Barriers to Universal Reasoning With Transformers (And How to Overcome Them)

Training-Free Universal Approximation by Prompting Random Transformers

Explaining Attention with Program Synthesis

Can Transformers Really Do It All? On the Compatibility of Inductive Biases Across Tasks

Emergence of Minimal Circuits for Indirect Object Identification in Attention-Only Transformers

Efficiently Representing Algorithms With Chain-of-Thought Transformers