arXiv:2609.15177v1 Announce Type: new
Abstract: Diffusion language models (dLLMs) promise fast inference by generating multiple tokens in parallel, but suffer severe performance degradation when para...
By Shijian Xu, Andrea Miele, Metod Jazbec, Volker Roth, Eric Nalisnick, Ilija Bogunovic
arXiv:2607. 01775v1 Announce Type: new Abstract: Discrete diffusion models have steadily improved in quality relative to autoregressive (AR) models.
By Marianne Arriola, Volodymyr Kuleshov
arXiv:2609. 04010v1 Announce Type: new Abstract: Large Language Models (LLMs) owe much of their success to next-token prediction (NTP), but their autoregressive (AR) structure requires slow, sequential token generation.
By Subham Sekhar Sahoo, Lingjie Chen, Khiem Pham, Jonathan Geuter, Chaitanya Dwivedi, Varad Pimpalkhute, Yash Akhauri, Alexander Moreno, Mikhail Yurochkin, Zhenting Wang, Mostafa Elhoushi, Nolan Dey, Shane Bergsma, Joel Hestness, John Thickstun, Eric Xing, Zhengzhong Liu
The paper introduces Clock Diffusion, a framework for semi‑autoregressive continuous diffusion language models that incorporates position‑dependent noise schedules, efficient training, and sampling algorithms. It presents two generation modes—block and sliding window—and defines ClockDLMs, a family of Gaussian models that achieve state‑of‑the‑art diffusion likelihoods on OpenWebText and outperform continuous baselines on GSM8K while matching or exceeding discrete diffusion models. The authors also propose Cache Grab, a set of efficient samplers that adapt accelerated inference techniques from discrete diffusion to further improve model quality and efficiency.
By Yair Schiff, Omer Belhasin, Roy Uziel, Matan Rusanovsky, Ran Zilberstein, Marianne Arriola, Gilad Turok, Guanghan Wang, Volodymyr Kuleshov, Michael Elad
PlaidQ is a 0.7B continuous diffusion language model designed for code generation. By distilling its iterative refinement trajectory into only a few denoising steps—or even a single step—PlaidQ achieves competitive performance with discrete diffusion models while dramatically reducing inference time. The study demonstrates that continuous diffusion can be effectively compressed, enabling efficient and accurate code generation with minimal computational overhead.
By Fred Zhangzhi Peng, Kaiwen Zheng, Anru R. Zhang
arXiv:2601. 07568v3 Announce Type: replace-cross Abstract: Diffusion large language models (dLLMs) offer capabilities beyond those of autoregressive (AR) LLMs, such as parallel decoding and random-order generation.
By Yu-Yang Qian, Junda Su, Lanxiang Hu, Peiyuan Zhang, Zhijie Deng, Peng Zhao, Hao Zhang