The paper introduces Less Uniform Diffusion (LUDI), a framework that improves uniform diffusion language models by using a less uniform loss and per-token time embeddings to guide reverse transitions and enable confidence-based few-step sampling. Experiments demonstrate that LUDI provides cleaner supervision, enhances few-step generation, and scales to a 7B model (LUDI-7B) that achieves a 3-token-per-step speedup over autoregressive decoding while matching masked diffusion baselines. The work suggests that UDLMs still have untapped potential for complex generation tasks.
By Kaibo Wang, Ding Ding, Fangyu Ding, Zijin Feng, Han Shi, Haili Bai, Jiacheng Sun, Yang Xiang
arXiv:2609.27220v1 Announce Type: new
Abstract: Diffusion language models generate text through iterative denoising, exposing intermediate trajectories before final answers are produced. We identify...
By Guoshenghui Zhao, Tan Yu, Weijie Zhao
CaLR: Causal Latent Revision for Robust Diffusion Reasoning proposes a framework that combines autoregressive and diffusion language models by reformulating reasoning as constrained latent optimization. The method uses a causal topology matrix from an expert model and implicit differentiation to perform gradient-guided thought revision, enforcing logical consistency and allowing dynamic self-correction of intermediate steps during parallel generation. Empirical results show CaLR achieves state‑of‑the‑art performance on complex benchmarks, surpassing strong autoregressive baselines and demonstrating superior robustness in constrained tasks such as Sudoku.
By Wei Cai, Jian Zhao, Yuchen Yuan, Xuelong Li
Draft Refinement Language Models (DRLMs) propose parallel revisions to a draft answer using a single denoiser that reconstructs every token from uniformly corrupted drafts, enabling both preservation and repair of tokens. The model evaluates each candidate draft by estimating a cavity energy from forward predictions, selecting revisions that lower this energy without a separate scoring model. Experiments on Countdown, Sudoku-Extreme, and Maze-Hard show DRLM outperforming autoregressive and discrete diffusion baselines, achieving 44.4% on Countdown-5 and 93.9% on Sudoku-Extreme with 6M parameters.
By Jaesang Yu, Seoyeon Byun, Yun Wang, Yan Lu, Juho Kim
arXiv:2606. 27617v1 Announce Type: cross Abstract: Masked Diffusion Models (MDMs) promise fast, parallel language generation, but their reverse transition factorises across token positions -- an approximation that breaks down in the few-step sampling regime where parallel generation ought to provide the greatest efficiency gains.
By Iskander Azangulov, Kianoosh Ashouritaklimi, Leo Zhang, Simon Vary, Patrick Rebeschini
arXiv:2510. 19990v2 Announce Type: replace Abstract: The reasoning paradigm, where language models reason before answering, has enabled breakthroughs on tasks such as mathematical problem-solving.
By Zachary Horvitz, Raghav Singhal, Hao Zou, Carles Domingo-Enrich, Zhou Yu, Rajesh Ranganath, Kathleen McKeown
The paper introduces a test‑time compute allocation framework for block diffusion language models, featuring Bounded Adaptive Confidence Decoding (BACD) for step‑wise adaptive denoising and the Think Coarse, Critic Fine (TCCF) paradigm for block‑wise generation. BACD dynamically adjusts denoising based on model confidence to accelerate inference while limiting error accumulation. TCCF allocates larger blocks for exploratory thinking and smaller blocks for precise refinement, with Progressive Block Size Extension stabilizing training across varying block sizes. Experiments on six benchmarks show the TDAR‑8B model achieves a 2.38× speedup and +3.4% average accuracy over the TraDo‑8B baseline.
By Yi Lu, Deyang Kong, Jianing Wang, Linsen Guo, Xue Wang, Qi Guo, Tao Gui, Xuanjing Huang, Wei Ye, Shikun Zhang, Wei Wang
arXiv:2601. 22954v2 Announce Type: replace-cross Abstract: Diffusion Large Language Models (dLLMs) have emerged as a promising alternative to purely autoregressive language models because they can decode multiple tokens in parallel.
By Yuezhou Hu, Harman Singh, Monishwaran Maheswaran, Haocheng Xi, Coleman Hooper, Jintao Zhang, Aditya Tomar, Michael W. Mahoney, Sewon Min, Mehrdad Farajtabar, Kurt Keutzer, Amir Gholami, Chenfeng Xu
The paper introduces Stitching Noisy Diffusion Thoughts, a self‑consistency framework that transforms low‑cost diffusion‑sampled reasoning into a reusable pool of step‑level candidates. By sampling many diverse trajectories with a masked diffusion language model, scoring each intermediate step with a process reward model, and stitching the highest‑quality steps across trajectories, the method recomputes only the final answer, separating exploration from evaluation. Experiments on math reasoning benchmarks show that step‑level recombination improves accuracy by up to 23.8% on six tasks while reducing latency by up to 1.8× compared to traditional diffusion models and unified architectures.
By Roy Miles, Aysim Toker, Andreea-Maria Oncescu, Jiankang Deng, Ismail Elezi
arXiv:2603. 01331v3 Announce Type: replace-cross Abstract: Discrete diffusion language models (dLLMs) generate text by iteratively denoising a masked sequence.
By Kejing Xia, Mingzhe Li, Lixuan Wei, Zhenbang Du, Xiangchi Yuan, Dachuan Shi, Qirui Jin, Wenke Lee
arXiv:2510. 13940v4 Announce Type: replace-cross Abstract: Recent progress in large language models (LLMs) has focused on test-time scaling to improve reasoning via increased inference computation, but often at the cost of efficiency.
By Zhen Yang, Mingyang Zhang, Feng Chen, Ganggui Ding, Liang Hou, Xin Tao, Ying-Cong Chen
arXiv:2604.14121v3 Announce Type: replace
Abstract: Large language models (LLMs) have become increasingly used for various tasks, often coupled with Chain-of-Thought (CoT) prompting to boost accuracy...
By Zipeng Ling, Shuliang Liu, Seonil Son, Shenghong Fu, Yuehao Tang, Yao Wan, Xuming Hu