arXiv Machine Learning September 11, 2026 By Zhihan Yang, Wei Guo, Shuibai Zhang, Subham Sekhar Sahoo, Yongxin Chen, Arash Vahdat, Morteza Mardani, John Thickstun

Continuous Diffusion Scales Competitively with Discrete Diffusion for Language

Read the original on arXiv Machine Learning →

The paper revisits the continuous diffusion language model Plaid and introduces RePlaid, aligning its architecture with modern discrete diffusion models. RePlaid achieves a compute gap of only 20× compared to autoregressive models, surpasses Duo with fewer parameters, and outperforms MDLM in over‑trained settings. On OpenWebText, RePlaid sets a new state‑of‑the‑art continuous diffusion perplexity of 22.1 and demonstrates superior generation quality, while theoretical analysis links likelihood‑based training to linear cross‑entropy over time and structured embedding geometries.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Jul 24

Understanding and Accelerating the Training of Masked Diffusion Language Models

arXiv:2605. 13026v2 Announce Type: replace-cross Abstract: Masked diffusion models (MDMs) have emerged as a promising alternative to autoregressive models (ARMs) for language modeling.

By Chunsan Hong, Sanghyun Lee, Chieh-Hsin Lai, Satoshi Hayakawa, Yuhta Takida, Yuki Mitsufuji, Seungryong Kim, Jong Chul Ye

llms diffusion benchmarks safety

arXiv Machine Learning

Jun 16

Semantic DLM+: Improving Diffusion Language Models through Bias-variance Trade-off in Transition Kernel Design

arXiv:2606. 15327v1 Announce Type: new Abstract: Diffusion Language Models (DLMs) have demonstrated strong scaling capacity as alternatives to autoregressive language models.

By Keyue Jiang, Yuxiang Wang, Yanan Zhao, Xiang Yu, Qifang Zhao, Bohan Tang, Baojian Zhou, Yanghua Xiao, Lin Qu, Xiaoxiao Xu

llms diffusion safety

arXiv AI

Jun 19

Diffusion Language Models: An Experimental Analysis

arXiv:2606. 19475v1 Announce Type: new Abstract: Large Language Models (LLMs) have revolutionized language modeling through autoregressive generation, enabling strong performance across a wide range of tasks.

By Thomas Bertolani, Davide Bucciarelli, Leonardo Zini, Marcella Cornia, Lorenzo Baraldi

llms diffusion benchmarks

arXiv Machine Learning

Jun 2

Consistent Diffusion Language Models

arXiv:2605. 00161v2 Announce Type: replace Abstract: Diffusion language models (DLMs) are an attractive alternative to autoregressive models because they promise sublinear-time, parallel generation, yet practical gains remain elusive as high-quality samples still demand hundreds of refinement steps.

By Hasan Amin, Yuan Gao, Yaser Souri, Subhojit Som, Ming Yin, Rajiv Khanna, Xia Song

llms diffusion efficiency

arXiv AI

Jun 2

IDLM: Inverse-distilled Diffusion Language Models

arXiv:2602. 19066v2 Announce Type: replace-cross Abstract: Diffusion Language Models (DLMs) have recently achieved strong results in text generation.

By David Li, Nikita Gushchin, Dmitry Abulkhanov, Eric Moulines, Ivan Oseledets, Maxim Panov, Alexander Korotin

llms diffusion efficiency

arXiv Machine Learning

Jun 18

Sumi: Open Uniform Diffusion Language Model from Scratch

arXiv:2606. 19005v1 Announce Type: cross Abstract: Diffusion models have become a promising alternative to autoregressive models.

By Mengyu Ye, Keito Kudo, Wataru Ikeda, Ryosuke Matsuda, Keisuke Sakaguchi, Jun Suzuki

llms diffusion benchmarks

One email a morning, machine-written

Related stories

Understanding and Accelerating the Training of Masked Diffusion Language Models

Semantic DLM+: Improving Diffusion Language Models through Bias-variance Trade-off in Transition Kernel Design

Diffusion Language Models: An Experimental Analysis

Consistent Diffusion Language Models

IDLM: Inverse-distilled Diffusion Language Models

Sumi: Open Uniform Diffusion Language Model from Scratch