arXiv:2609.37974v1 Announce Type: cross
Abstract: Masked diffusion models (MDMs) generate text by unmasking several tokens per step, but they are trained and sampled under different conditions. The m...
By Manuel Madeira, Amitis Shidani, Alice Bizeul, Victor Turrisi, Louis B\'ethune, Bhavika Devnani, Dan Busbridge, Pierre Ablin, Jo\~ao Monteiro
arXiv:2610.17266v1 Announce Type: new
Abstract: Diffusion language models are a promising and more efficient alternative to autoregressive models. However, most RL post-training techniques are not ea...
By Vaibhav Singh, Eugene Belilovsky
arXiv:2607. 15200v1 Announce Type: cross Abstract: Reinforcement learning has proven effective for improving reasoning in large language models, but extending it to Masked Diffusion Language Models (MDLMs) remains challenging due to the intractability of the log-likelihood estimation.
By Haran Raajesh, Kulin Shah, Adam Klivans, Philipp Kr\"ahenb\"uhl
The paper introduces Prefix-Block Policy Optimization (PBPO), a method that adapts block-level policy optimization to Diffusion Large Language Models (dLLMs). PBPO constructs a prefix-block state and uses a single model evaluation to create a factorized one‑step surrogate for block likelihood, allowing surrogate importance weighting and clipping to be applied separately to each token. Experiments on LLaDA-family models across six benchmarks and three generation lengths show that PBPO achieves competitive performance compared to existing dLLM reinforcement learning methods.
By Katherine Tieu, Dongqi Fu, Si Zhang, Hanqing Zeng, Tianxin Wei, Chenglin Wei, Yinglong Xia, Hong Li, Jingrui He
arXiv:2604. 18738v3 Announce Type: replace Abstract: Diffusion language models (dLLMs) generate text through iterative denoising, filling multiple masked positions at each step.
By Lin Yao
arXiv:2610.22543v1 Announce Type: new
Abstract: Diffusion language models (DLMs) promise parallel text generation, but in practice face an unresolved speed-quality trade-off. Exact decoding fills in...
By Eliezer Shlomi, Eliron Rahimi, Avi Mendelson, Chaim Baskin
arXiv:2606. 02544v1 Announce Type: cross Abstract: Diffusion large language models (dLLMs) have recently emerged as a promising alternative to autoregressive (AR) LLMs, offering faster inference through parallel or blockwise decoding.
By Junxia Cui, Haotian Ye, Runchu Tian, Hongcan Guo, Jinya Jiang, Haoru Li, Chaojie Ren, Yiming Huang, Kaijie Zhu, Zhongkai Yu, Kun Zhou, Jingbo Shang
The paper introduces TIPS, a tiny inference-time policy that controls which tokens are visible during each denoising pass of masked diffusion language models. TIPS, a 600‑K‑parameter shallow transformer, learns to unmask and remask tokens while keeping the language model frozen, using visibility‑aware confidence features and a fixed schedule for expected visibility. Trained via supervised warmup and reinforcement learning, TIPS achieves over 58% accuracy on GSM8K and MATH‑500 with only 32 decoder evaluations, surpassing strong baselines by more than 11 percentage points and remaining competitive at higher budgets without retraining.
By Ya-Shi Zhang, Jian Tang, Pierre-Andr\'e No\"el, Oleksiy Ostapenko
arXiv:2608. 13925v1 Announce Type: cross Abstract: Diffusion large language models (dLLMs) accelerate language generation by predicting multiple masks in a single forward pass.
By Yuji Ren, Chenkai Xu, Zhuocheng Gong, Jianguo Li, Zhijie Deng
CoDR (Confidence Drift Remasking) is a training‑free, sampler‑agnostic refinement pass for masked diffusion language models that detects and corrects early token commitments that become unsupported as context grows denser. It estimates confidence drift for all committed positions using only a few forward passes and remasks only the tokens whose confidence has dropped, thereby improving accuracy across multiple backbones, tasks, and samplers with modest computational overhead.
By Yue Wu, Qinghe Zhang, Yu Zhang, Jian Huang
arXiv:2606. 12232v1 Announce Type: new Abstract: Masked diffusion language models (dLLMs) have recently emerged as a competitive alternative to autoregressive language models, with the promise of faster inference via parallel token generation.
By Stipe Frkovic, Metod Jazbec, Dan Zhang, Christian A. Naesseth, Ilija Bogunovic, Eric Nalisnick
arXiv:2610.29255v1 Announce Type: new
Abstract: Masked diffusion language models have emerged as an alternative to autoregressive models, enabling parallel token generation. However, standard masked...
By Parul Gupta, Satwik, Avni Aggarwal, Bhavesh Gurnani, Arnab Kumar Mondal, Dinesh Khandelwal, Dinesh Garg, Parag Singla