arXiv AI July 20, 2026 By Andy Catruna, Emilian Radoi

Induction in Both Directions: A Mechanistic Analysis of In-Context Learning in Masked Diffusion Language Models

arXiv:2607. 15893v1 Announce Type: cross Abstract: While the internal mechanisms of autoregressive (AR) transformers have been studied extensively, much less is known about diffusion language models (DLMs), an emerging alternative that generates text by iterative denoising.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv AI

Jun 8

Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillation

arXiv:2606. 06712v1 Announce Type: cross Abstract: We study the transformation of autoregressive models (ARLMs) into diffusion language models (DLMs).

By Xingyu Su, Jacob Helwig, Shubham Parashar, Atharv Chagi, Lakshmi Jotsna, Degui Zhi, James Caverlee, Dileep Kalathil, Shuiwang Ji

llms diffusion efficiency

arXiv Machine Learning

Jun 3

Learning without training: The implicit dynamics of in-context learning

arXiv:2507. 16003v4 Announce Type: replace-cross Abstract: One of the most striking features of Large Language Models (LLMs) is their ability to learn in-context.

By Benoit Dherin, Michael Munn, Hanna Mazzawi, Michael Wunder, Javier Gonzalvo

llms

arXiv AI

Jul 24

Understanding and Accelerating the Training of Masked Diffusion Language Models

arXiv:2605. 13026v2 Announce Type: replace-cross Abstract: Masked diffusion models (MDMs) have emerged as a promising alternative to autoregressive models (ARMs) for language modeling.

By Chunsan Hong, Sanghyun Lee, Chieh-Hsin Lai, Satoshi Hayakawa, Yuhta Takida, Yuki Mitsufuji, Seungryong Kim, Jong Chul Ye

llms diffusion benchmarks safety

arXiv AI

Jun 2

SimSD: Simple Speculative Decoding in Diffusion Language Models

arXiv:2606. 02544v1 Announce Type: cross Abstract: Diffusion large language models (dLLMs) have recently emerged as a promising alternative to autoregressive (AR) LLMs, offering faster inference through parallel or blockwise decoding.

By Junxia Cui, Haotian Ye, Runchu Tian, Hongcan Guo, Jinya Jiang, Haoru Li, Chaojie Ren, Yiming Huang, Kaijie Zhu, Zhongkai Yu, Kun Zhou, Jingbo Shang

llms diffusion efficiency benchmarks

arXiv Machine Learning

Jun 5

Masks Can Be Distracting: On Context Comprehension in Diffusion Language Models

arXiv:2511. 21338v2 Announce Type: replace Abstract: Masked Diffusion Language Models (MDLMs) have recently emerged as a promising alternative to Autoregressive Language Models (ARLMs), leveraging a denoising objective that, in principle, should enable more uniform context utilisation.

By Julianna Piskorz, Cristina Pinneri, Alvaro Correia, Motasem Alfarra, Risheek Garrepalli, Christos Louizos

llms diffusion fine-tuning safety

arXiv AI

Jul 7

Training Hybrid Block Diffusion Language Models with Partial Bidirectionality

arXiv:2607. 02805v1 Announce Type: cross Abstract: High-throughput long-context generation is one of the central challenges for large language models.

By Pranshu Chaturvedi, Parth Shroff, Tarun Suresh, Hangoo Kang, Kaiyue Wen

llms diffusion efficiency

Related stories

Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillation

Learning without training: The implicit dynamics of in-context learning

Understanding and Accelerating the Training of Masked Diffusion Language Models

SimSD: Simple Speculative Decoding in Diffusion Language Models

Masks Can Be Distracting: On Context Comprehension in Diffusion Language Models

Training Hybrid Block Diffusion Language Models with Partial Bidirectionality