The paper introduces Less Uniform Diffusion (LUDI), a framework that improves uniform diffusion language models by using a less uniform loss and per-token time embeddings to guide reverse transitions and enable confidence-based few-step sampling. Experiments demonstrate that LUDI provides cleaner supervision, enhances few-step generation, and scales to a 7B model (LUDI-7B) that achieves a 3-token-per-step speedup over autoregressive decoding while matching masked diffusion baselines. The work suggests that UDLMs still have untapped potential for complex generation tasks.
By Kaibo Wang, Ding Ding, Fangyu Ding, Zijin Feng, Han Shi, Haili Bai, Jiacheng Sun, Yang Xiang
arXiv:2609.27220v1 Announce Type: new
Abstract: Diffusion language models generate text through iterative denoising, exposing intermediate trajectories before final answers are produced. We identify...
By Guoshenghui Zhao, Tan Yu, Weijie Zhao
CaLR: Causal Latent Revision for Robust Diffusion Reasoning proposes a framework that combines autoregressive and diffusion language models by reformulating reasoning as constrained latent optimization. The method uses a causal topology matrix from an expert model and implicit differentiation to perform gradient-guided thought revision, enforcing logical consistency and allowing dynamic self-correction of intermediate steps during parallel generation. Empirical results show CaLR achieves state‑of‑the‑art performance on complex benchmarks, surpassing strong autoregressive baselines and demonstrating superior robustness in constrained tasks such as Sudoku.
By Wei Cai, Jian Zhao, Yuchen Yuan, Xuelong Li
Draft Refinement Language Models (DRLMs) propose parallel revisions to a draft answer using a single denoiser that reconstructs every token from uniformly corrupted drafts, enabling both preservation and repair of tokens. The model evaluates each candidate draft by estimating a cavity energy from forward predictions, selecting revisions that lower this energy without a separate scoring model. Experiments on Countdown, Sudoku-Extreme, and Maze-Hard show DRLM outperforming autoregressive and discrete diffusion baselines, achieving 44.4% on Countdown-5 and 93.9% on Sudoku-Extreme with 6M parameters.
By Jaesang Yu, Seoyeon Byun, Yun Wang, Yan Lu, Juho Kim
arXiv:2606. 27617v1 Announce Type: cross Abstract: Masked Diffusion Models (MDMs) promise fast, parallel language generation, but their reverse transition factorises across token positions -- an approximation that breaks down in the few-step sampling regime where parallel generation ought to provide the greatest efficiency gains.
By Iskander Azangulov, Kianoosh Ashouritaklimi, Leo Zhang, Simon Vary, Patrick Rebeschini
arXiv:2510. 19990v2 Announce Type: replace Abstract: The reasoning paradigm, where language models reason before answering, has enabled breakthroughs on tasks such as mathematical problem-solving.
By Zachary Horvitz, Raghav Singhal, Hao Zou, Carles Domingo-Enrich, Zhou Yu, Rajesh Ranganath, Kathleen McKeown