arXiv AI

Blurring the Block Boundaries: Overlapping Block Denoising for Efficient Diffusion Language Models

The paper introduces Blurring the Block Boundaries (B3), a technique that allows two blocks in a diffusion language model to be refined simultaneously during a single forward pass. B3 dynamically opens the next block only when the model is ready, eliminating the need for hyperparameters or additional training. Experiments on ten datasets show that B3 maintains generation quality while achieving an 11.5–14.5% speedup over baseline models.

Hugging Face Trending Papers
Jul 20

FlowBlock: Wavefront-Parallel Decoding for Self-Correcting Diffusion Language Models

Block-wise diffusion large language models (dLLMs) decode sequentially at the block level, enabling effective KV-cache reuse across blocks but making inter-block decoding strictly serial. Prior work has attempted to unlock inter-block parallelism through post-training methods, but achieves only modest speedups and often degrades accuracy.

arXiv Machine Learning
Jun 26

Dynamic-dLLM: Dynamic Cache-Budget and Adaptive Parallel Decoding for Training-Free Acceleration of Diffusion LLM

arXiv:2606. 26120v1 Announce Type: cross Abstract: Diffusion Large Language Models (dLLMs) offer a promising alternative to autoregressive models, excelling in text generation tasks due to their bidirectional attention mechanisms.

By Tianyi Wu, Xiaoxi Sun, Yanhua Jiao, Yulin Li, Yixin Chen, YunHao Cao, YiQi Hu, Zhuotao Tian
arXiv Machine Learning
Jun 30

Multi-Block Diffusion Language Models

arXiv:2606. 29215v1 Announce Type: new Abstract: Block Diffusion Language Models (BD-LMs) improve diffusion-based text generation with KV caching and flexible-length generation.

By Yijie Jin, Jiajun Xu, Yuxuan Liu, Chenkai Xu, Yi Tu, Jiajun Li, Dandan Tu, Xiaohui Yan, Kai Yu, Pengfei Liu, Zhijie Deng
arXiv Machine Learning
Nov 6

PBPO: Prefix-Block Policy Optimization for Diffusion Language Models

The paper introduces Prefix-Block Policy Optimization (PBPO), a method that adapts block-level policy optimization to Diffusion Large Language Models (dLLMs). PBPO constructs a prefix-block state and uses a single model evaluation to create a factorized one‑step surrogate for block likelihood, allowing surrogate importance weighting and clipping to be applied separately to each token. Experiments on LLaDA-family models across six benchmarks and three generation lengths show that PBPO achieves competitive performance compared to existing dLLM reinforcement learning methods.

By Katherine Tieu, Dongqi Fu, Si Zhang, Hanqing Zeng, Tianxin Wei, Chenglin Wei, Yinglong Xia, Hong Li, Jingrui He
arXiv AI
Jun 15

Residual Context Diffusion Language Models

arXiv:2601. 22954v2 Announce Type: replace-cross Abstract: Diffusion Large Language Models (dLLMs) have emerged as a promising alternative to purely autoregressive language models because they can decode multiple tokens in parallel.

By Yuezhou Hu, Harman Singh, Monishwaran Maheswaran, Haocheng Xi, Coleman Hooper, Jintao Zhang, Aditya Tomar, Michael W. Mahoney, Sewon Min, Mehrdad Farajtabar, Kurt Keutzer, Amir Gholami, Chenfeng Xu