arXiv AI July 20, 2026 By Andy Catruna, Emilian Radoi

Induction in Both Directions: A Mechanistic Analysis of In-Context Learning in Masked Diffusion Language Models

arXiv:2607. 15893v1 Announce Type: cross Abstract: While the internal mechanisms of autoregressive (AR) transformers have been studied extensively, much less is known about diffusion language models (DLMs), an emerging alternative that generates text by iterative denoising.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Machine Learning

Sep 16

Window-Diffusion: Accelerating Diffusion Language Model Inference with Windowed Token Pruning and Caching

The paper introduces Window-Diffusion, a method that accelerates diffusion language model inference by pruning and caching tokens within a sliding window. It categorizes undecoded tokens into active, buffer, and far-field groups, computing only the first two while discarding the rest. Experiments on LLaDA and Dream demonstrate up to 99× speedup with minimal loss in generation quality.

By Fengrui Zuo, Zhiwei Ke, Yiming Liu, Wenqi Lou, Chao Wang, Xuehai Zhou

llms diffusion efficiency

arXiv AI

Jun 8

Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillation

arXiv:2606. 06712v1 Announce Type: cross Abstract: We study the transformation of autoregressive models (ARLMs) into diffusion language models (DLMs).

By Xingyu Su, Jacob Helwig, Shubham Parashar, Atharv Chagi, Lakshmi Jotsna, Degui Zhi, James Caverlee, Dileep Kalathil, Shuiwang Ji

llms diffusion efficiency

arXiv AI

Sep 1

DIP: Dynamic In-Context Planner For Diffusion Language Models

arXiv:2601.03199v2 Announce Type: replace-cross Abstract: Diffusion language models (DLMs) have shown strong potential for general natural language tasks with in-context examples. Existing In-Context...

By Yang Li, Han Meng, Chenan Wang, Zhenyu Bi, Xuan Wang, Haipeng Chen

llms diffusion benchmarks

arXiv Machine Learning

Jun 3

Learning without training: The implicit dynamics of in-context learning

arXiv:2507. 16003v4 Announce Type: replace-cross Abstract: One of the most striking features of Large Language Models (LLMs) is their ability to learn in-context.

By Benoit Dherin, Michael Munn, Hanna Mazzawi, Michael Wunder, Javier Gonzalvo

llms

arXiv AI

Jul 24

Understanding and Accelerating the Training of Masked Diffusion Language Models

arXiv:2605. 13026v2 Announce Type: replace-cross Abstract: Masked diffusion models (MDMs) have emerged as a promising alternative to autoregressive models (ARMs) for language modeling.

By Chunsan Hong, Sanghyun Lee, Chieh-Hsin Lai, Satoshi Hayakawa, Yuhta Takida, Yuki Mitsufuji, Seungryong Kim, Jong Chul Ye

llms diffusion benchmarks safety

arXiv Computation and Language

Aug 21

Remask, Don't Replace: Token-to-Mask Refinement in Diffusion Language Models

arXiv:2604. 18738v3 Announce Type: replace Abstract: Diffusion language models (dLLMs) generate text through iterative denoising, filling multiple masked positions at each step.

By Lin Yao

llms diffusion

One email a morning, machine-written

Related stories

Window-Diffusion: Accelerating Diffusion Language Model Inference with Windowed Token Pruning and Caching

Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillation

DIP: Dynamic In-Context Planner For Diffusion Language Models

Learning without training: The implicit dynamics of in-context learning

Understanding and Accelerating the Training of Masked Diffusion Language Models

Remask, Don't Replace: Token-to-Mask Refinement in Diffusion Language Models