Informed Masking (IM) is a new technique for aligning Diffusion Large Language Models (dLLMs) with Reinforcement Learning (RL). It identifies a systematic upstream/downstream token structure in dLLM rollouts and shows that masking downstream tokens creates better subproblems for likelihood estimation. When integrated into three state‑of‑the‑art dLLM RL methods on LLaDA‑8B‑Instruct, IM yields up to 2.01%, 8.68%, and 5.77% relative average gains on math and planning benchmarks while improving training stability.
By Xiaoyi Yu, Enver Sangineto, Pei Fu, Fiorenzo Parascandolo, Wenhui Tan, Ruikang Zhang, Rita Cucchiara, Ruihua Song, Jian Luan
arXiv:2604. 26985v2 Announce Type: replace-cross Abstract: Masked diffusion models (MDMs) generate discrete sequences by iterative denoising under an absorbing masking process.
By Michael Cardei, Huu Binh Ta, Ferdinando Fioretto
arXiv:2512. 09106v4 Announce Type: replace Abstract: Diffusion (Large) Language Models (dLLMs) now match the downstream performance of their autoregressive counterparts on many tasks, while holding the promise of being more efficient during inference.
By Metod Jazbec, Theo X. Olausson, Louis B\'ethune, Pierre Ablin, Michael Kirchhof, Jo\~ao Monteiro, Victor Turrisi, Jason Ramapuram, Marco Cuturi
arXiv:2609.37974v1 Announce Type: cross
Abstract: Masked diffusion models (MDMs) generate text by unmasking several tokens per step, but they are trained and sampled under different conditions. The m...
By Manuel Madeira, Amitis Shidani, Alice Bizeul, Victor Turrisi, Louis B\'ethune, Bhavika Devnani, Dan Busbridge, Pierre Ablin, Jo\~ao Monteiro
The paper introduces Learned Relay Representations (Relay), a technique for Masked Diffusion Models (MDMs) that preserves and forwards internal latent information across denoising steps via a differentiable per-token channel trained with truncated backpropagation through time. Relay enables MDMs to be forward‑thinking, avoiding costly recomputation of internal representations and improving efficiency. Applied to Fast‑dLLM v2, Relay outperforms standard supervised fine‑tuning on coding tasks and reduces inference latency by up to 32%, demonstrating a clear performance‑latency advantage for diffusion language models.
By Benjamin Rozonoyer, Jacopo Minniti, Dhruvesh Patel, Neil Band, Avishek Joey Bose, Tim G. J. Rudner, Andrew McCallum
arXiv:2607. 15200v1 Announce Type: cross Abstract: Reinforcement learning has proven effective for improving reasoning in large language models, but extending it to Masked Diffusion Language Models (MDLMs) remains challenging due to the intractability of the log-likelihood estimation.
By Haran Raajesh, Kulin Shah, Adam Klivans, Philipp Kr\"ahenb\"uhl
arXiv:2606. 29275v1 Announce Type: new Abstract: Diffusion Language Models (DLMs) are typically trained under fixed context structures, restricting denoising to predetermined token subsets.
By Gagan Jain
arXiv:2607. 19686v1 Announce Type: cross Abstract: Masked diffusion models (MDMs) are a promising family of language generators, but achieving high-quality few-step generation remains challenging.
By Sijin Chen, Yinuo Ren, Heyang Zhao, Ziheng Cheng, Quanquan Gu, Lexing Ying
arXiv:2602. 18695v2 Announce Type: replace Abstract: Existing insertion-based masked diffusion models that generate sequences by interleaving token insertion with unmasking use fixed schedules that are not dependent on the data.
By Dhruvesh Patel, Benjamin Rozonoyer, Gaurav Pandey, Tahira Naseem, Ram\'on Fernandez Astudillo, Andrew McCallum
arXiv:2604. 24357v2 Announce Type: replace-cross Abstract: Diffusion language models generate without a fixed left-to-right order, leaving token ordering as a central algorithmic choice.
By Dake Bu, Wei Huang, Andi Han, Hau-San Wong, Qingfu Zhang, Taiji Suzuki, Atsushi Nitanda
arXiv:2601. 22450v2 Announce Type: replace-cross Abstract: Masked Diffusion Language Models have recently emerged as a powerful generative paradigm, yet their generalization properties remain understudied compared to their auto-regressive counterparts.
By Jianhao Huang, Baharan Mirzasoleiman
arXiv:2602. 11590v3 Announce Type: replace Abstract: Masked diffusion models (MDMs) have emerged as a promising alternative to autoregressive models, enabling parallel token generation while achieving competitive performance.
By Yair Schiff, Omer Belhasin, Roy Uziel, Guanghan Wang, Marianne Arriola, Gilad Turok, Ran Zilberstein, Michael Elad, Volodymyr Kuleshov