arXiv Computation and Language

Register Tokens for Bounded-State Reasoning in Diffusion Language Models

arXiv AI

Jun 15

Residual Context Diffusion Language Models

arXiv:2601. 22954v2 Announce Type: replace-cross Abstract: Diffusion Large Language Models (dLLMs) have emerged as a promising alternative to purely autoregressive language models because they can decode multiple tokens in parallel.

By Yuezhou Hu, Harman Singh, Monishwaran Maheswaran, Haocheng Xi, Coleman Hooper, Jintao Zhang, Aditya Tomar, Michael W. Mahoney, Sewon Min, Mehrdad Farajtabar, Kurt Keutzer, Amir Gholami, Chenfeng Xu

llms diffusion benchmarks

Hugging Face Trending Papers

Jun 3

SAID: Accelerating Diffusion-Based Language Models via Scaffold-Aware Iterative Decoding

Diffusion large language models (DLLMs) enable non-autoregressive generation by iteratively denoising corrupted token sequences with bidirectional context. Despite their ability to update multiple positions in parallel, inference remains costly due to the many denoising steps required for high-quality generation.

llms diffusion benchmarks

arXiv Machine Learning

Sep 16

Window-Diffusion: Accelerating Diffusion Language Model Inference with Windowed Token Pruning and Caching

The paper introduces Window-Diffusion, a method that accelerates diffusion language model inference by pruning and caching tokens within a sliding window. It categorizes undecoded tokens into active, buffer, and far-field groups, computing only the first two while discarding the rest. Experiments on LLaDA and Dream demonstrate up to 99× speedup with minimal loss in generation quality.

By Fengrui Zuo, Zhiwei Ke, Yiming Liu, Wenqi Lou, Chao Wang, Xuehai Zhou

llms diffusion efficiency

arXiv AI

Jun 2

SimSD: Simple Speculative Decoding in Diffusion Language Models

arXiv:2606. 02544v1 Announce Type: cross Abstract: Diffusion large language models (dLLMs) have recently emerged as a promising alternative to autoregressive (AR) LLMs, offering faster inference through parallel or blockwise decoding.

By Junxia Cui, Haotian Ye, Runchu Tian, Hongcan Guo, Jinya Jiang, Haoru Li, Chaojie Ren, Yiming Huang, Kaijie Zhu, Zhongkai Yu, Kun Zhou, Jingbo Shang

llms diffusion efficiency benchmarks

arXiv AI

Aug 25

CAI-DLLM: Convergence Aware Inference for Diffusion Language Models

arXiv:2608.22646v1 Announce Type: new Abstract: Diffusion language models can generate many tokens in parallel, but they still require repeated denoising steps during inference. This makes generation...

By Farhana Amin, Sabiha Afroz, Dimitrios S. Nikolopoulos

llms diffusion benchmarks

arXiv AI

Jun 4

Supportive Token Revealing for Fast Diffusion Language Model Decoding

arXiv:2606. 04236v1 Announce Type: cross Abstract: Discrete diffusion language models can generate text efficiently by updating multiple masked positions in parallel, but this parallelism introduces a quality-latency trade-off.

By Giries Abu Ayoub, Mario Barbara, Llu\'is Pastor-P\'erez, Tanja Bien, Aneesh Barthakur, Alaa Maalouf, Loay Mualem

llms diffusion benchmarks

arXiv Machine Learning

Jun 11

FOCUS: DLLMs Know How to Tame Their Compute Bound

arXiv:2601. 23278v2 Announce Type: replace Abstract: Diffusion Large Language Models (DLLMs) offer a compelling alternative to Auto-Regressive models, but their deployment is constrained by high decoding cost.

By Kaihua Liang, Xin Tan, An Zhong, Hong Xu, Marco Canini

llms diffusion efficiency benchmarks

arXiv Machine Learning

Jul 20

Adaptive Multi-Step Lookahead Decoding for Diffusion Language Models

arXiv:2607. 15655v1 Announce Type: cross Abstract: Masked diffusion language models (DLMs) enable parallel text generation by iteratively refining masked tokens, offering a promising alternative to autoregressive decoding.

By Yingqian Cui, Wei Deng, Lantao Mei, Hang Li, Charu C. Aggarwal, Hui Liu, Yue Xing

llms diffusion benchmarks

arXiv Machine Learning

Jun 2

d2: Improving Reasoning in Diffusion Language Models via Trajectory Likelihood Estimation

arXiv:2509. 21474v4 Announce Type: replace Abstract: While diffusion language models (DLMs) have achieved competitive performance in text generation, improving their reasoning ability with reinforcement learning remains an active research area.

By Guanghan Wang, Gilad Turok, Yair Schiff, Marianne Arriola, Volodymyr Kuleshov

llms diffusion reinforcement-learning benchmarks

arXiv AI

Jul 14

MetaState: Persistent Working Memory Enhances Reasoning in Discrete Diffusion Language Models

arXiv:2603. 01331v3 Announce Type: replace-cross Abstract: Discrete diffusion language models (dLLMs) generate text by iteratively denoising a masked sequence.

By Kejing Xia, Mingzhe Li, Lixuan Wei, Zhenbang Du, Xiangchi Yuan, Dachuan Shi, Qirui Jin, Wenke Lee

llms diffusion benchmarks

arXiv Machine Learning

Jun 5

A Survey on Diffusion Language Models

arXiv:2508. 10875v3 Announce Type: replace-cross Abstract: Diffusion Language Models (DLMs) are rapidly emerging as a powerful and promising alternative to the dominant autoregressive (AR) paradigm.

By Tianyi Li, Mingda Chen, Bowei Guo, Zhiqiang Shen

llms diffusion nlp efficiency multimodal benchmarks

arXiv AI

Jul 17

Mask-Aware Policy Gradients for Diffusion Language Models

arXiv:2607. 15200v1 Announce Type: cross Abstract: Reinforcement learning has proven effective for improving reasoning in large language models, but extending it to Masked Diffusion Language Models (MDLMs) remains challenging due to the intractability of the log-likelihood estimation.

By Haran Raajesh, Kulin Shah, Adam Klivans, Philipp Kr\"ahenb\"uhl

llms diffusion reinforcement-learning benchmarks

Related stories