The paper studies how the order in which tokens are committed in masked diffusion language models affects accuracy. It finds that when the final answer is committed before the preceding reasoning (an answer‑first trajectory), accuracy can suffer compared to unrestricted decoding, especially on tasks like GSM8K and MATH‑500. Experiments with controlled token positions show that delaying the answer token can improve performance, indicating that commitment order influences the context and output allocation of the model.
By Jewon Yeom, Jaewon Sok, Seonghyeon Park, Jeongjae Park, Hwiyeong Lee, Taesup Kim
arXiv:2606. 14620v1 Announce Type: new Abstract: Open diffusion language models are marketed as parallel, non-autoregressive decoders, yet the order in which a shipped checkpoint actually commits its tokens is almost never measured.
By Ali Asaria, Tony Salomone, Deep Gandhi
arXiv:2608.30922v1 Announce Type: new
Abstract: Masked diffusion language models predict tokens from a partially observed response canvas, enabling bidirectional conditioning and parallel token refin...
By Wail Bouhedja, Amr Mohamed, Guokan Shang
Masked diffusion language models predict tokens from a partially observed response canvas, enabling bidirectional conditioning and parallel token refinement. Yet standard masked-diffusion decoders use...
arXiv:2605.11467v2 Announce Type: replace-cross
Abstract: Reasoning models post-hoc rationalize answers they have already committed to internally, producing chains of *reasoning theater*: deliberativ...
By Swapnil Parekh, Naman Goyal
arXiv:2607. 28166v2 Announce Type: replace-cross Abstract: Diffusion language models expose a provisional prediction at every denoising step, and on many tasks the candidate answer inside it stabilizes before the step schedule is exhausted.
By Chia-Ming Lee, Shao-Kai Liu, Ming-Ching Chang, Xin Li, Yu-Lun Liu, Chih-Chung Hsu
arXiv:2601. 15165v4 Announce Type: replace-cross Abstract: Diffusion Large Language Models (dLLMs) break the rigid left-to-right constraint of traditional LLMs, enabling token generation in arbitrary orders.
By Zanlin Ni, Shenzhi Wang, Yang Yue, Tianyu Yu, Weilin Zhao, Yeguo Hua, Tianyi Chen, Jun Song, Cheng Yu, Bo Zheng, Gao Huang
arXiv:2609.39560v1 Announce Type: new
Abstract: Sampling several responses and voting over their answers can improve a language model's accuracy, but repeated answers limit the benefit of additional...
By Michael Helcig, Martin Jaggi
arXiv:2609.38536v1 Announce Type: cross
Abstract: Diffusion-based large language models (dLLMs) promise to break the sequential latency bottleneck of autoregressive agents through parallel decoding,...
By Jiacheng Qiu, Christopher E. Mower, Jan Peters, Haitham Bou-Ammar, Matthieu Zimmer
arXiv:2609.32964v2 Announce Type: replace
Abstract: Language models (LMs) often hallucinate by committing to confident answers rather than abstaining, even when they do not have enough information to...
By Vy Nguyen, Ziqi Xu, Jeffrey Chan, Estrid He, Feng Xia, Renqiang Luo, Erik Cambria, Xiuzhen Zhang
The paper introduces committed reveal sampling (CRS), a training‑free sampler for uniform discrete diffusion models that stores selected argmax tokens as persistent context for subsequent predictions. CRS keeps these tokens visible in later model inputs, which theoretically prevents Bayes error from increasing as noise decreases and encourages consistent sequence‑level choices. Empirical tests on Duo‑distilled data show that CRS without top‑p truncation achieves lower generative perplexity than fixed‑p baselines across various numbers of function evaluations, offering a more favorable perplexity–entropy trade‑off.
By Satoshi Hayakawa
arXiv:2606. 20560v1 Announce Type: cross Abstract: LLM reasoning transparency is a critical affordance for understanding model decisions, mitigating misuse and misalignment, and debugging surprising model behaviors.
By Joshua Engels, Callum McDougall, Bilal Chughtai, Janos Kramar, Senthoran Rajamanoharan, Cindy Wu, Arthur Conmy, Asic Q Chen, Jean Tarbouriech, Min Ma, Brendan O'Donoghue, Jo\~ao Gabriel Lopes de Oliveira, Rohin Shah, Neel Nanda