arXiv AI October 1, 2026 By Loay Mualem, Llu\'is Pastor-P\'erez, Vinh Tong, Andrei Manolache, Tanja Bien, Steffen Staab, Mathias Niepert

Fine-Tuning Diffusion Language Models with Context Selection and Target Weighting

Read the original on arXiv AI →

The Flow has not summarised this story yet — read it at arXiv AI.

arXiv Machine Learning

Jun 25

Improved Large Language Diffusion Models

arXiv:2606. 25331v1 Announce Type: cross Abstract: Modern large language models are predominantly trained with autoregressive factorization and causal attention.

By Shen Nie, Qiyang Min, Shaoxuan Xu, Zihao Huang, Yuxuan Song, Yong Shan, Yankai Lin, Wayne Xin Zhao, Chongxuan Li, Ji-Rong Wen

llms diffusion fine-tuning benchmarks

arXiv Machine Learning

Jun 5

Masks Can Be Distracting: On Context Comprehension in Diffusion Language Models

arXiv:2511. 21338v2 Announce Type: replace Abstract: Masked Diffusion Language Models (MDLMs) have recently emerged as a promising alternative to Autoregressive Language Models (ARLMs), leveraging a denoising objective that, in principle, should enable more uniform context utilisation.

By Julianna Piskorz, Cristina Pinneri, Alvaro Correia, Motasem Alfarra, Risheek Garrepalli, Christos Louizos

llms diffusion fine-tuning safety

arXiv AI

Jun 4

Supportive Token Revealing for Fast Diffusion Language Model Decoding

arXiv:2606. 04236v1 Announce Type: cross Abstract: Discrete diffusion language models can generate text efficiently by updating multiple masked positions in parallel, but this parallelism introduces a quality-latency trade-off.

By Giries Abu Ayoub, Mario Barbara, Llu\'is Pastor-P\'erez, Tanja Bien, Aneesh Barthakur, Alaa Maalouf, Loay Mualem

llms diffusion benchmarks

arXiv AI

Jul 17

Mask-Aware Policy Gradients for Diffusion Language Models

arXiv:2607. 15200v1 Announce Type: cross Abstract: Reinforcement learning has proven effective for improving reasoning in large language models, but extending it to Masked Diffusion Language Models (MDLMs) remains challenging due to the intractability of the log-likelihood estimation.

By Haran Raajesh, Kulin Shah, Adam Klivans, Philipp Kr\"ahenb\"uhl

llms diffusion reinforcement-learning benchmarks

Hugging Face Trending Papers

Jun 10

Beyond Fully Random Masking: Attention-Guided Denoising and Optimization for Diffusion Language Models

Diffusion large language models (dLLMs) offer an efficient alternative to autoregressive models through parallel decoding, yet existing post-training methods largely rely on random masking strategies that overlook intrinsic token dependencies. In this work, we present an empirical analysis of attention in dLLMs and show that tokens attending more strongly to unmasked context exhibit greater generation stability and play a critical role in reasoning.

llms diffusion reinforcement-learning fine-tuning benchmarks

arXiv Computation and Language

Aug 21

Remask, Don't Replace: Token-to-Mask Refinement in Diffusion Language Models

arXiv:2604. 18738v3 Announce Type: replace Abstract: Diffusion language models (dLLMs) generate text through iterative denoising, filling multiple masked positions at each step.

By Lin Yao

llms diffusion