arXiv AI
Jul 7
Training Hybrid Block Diffusion Language Models with Partial Bidirectionality
arXiv:2607. 02805v1 Announce Type: cross Abstract: High-throughput long-context generation is one of the central challenges for large language models.
DiffuMamba: High-Throughput Diffusion LMs with Mamba Backbone
arXiv:2511. 15927v4 Announce Type: replace-cross Abstract: Diffusion language models (DLMs) have emerged as a promising alternative to autoregressive (AR) generation, yet their reliance on Transformer backbones limits inference efficiency due to quadratic attention or KV-cache overhead.
Related stories
arXiv AI
Jun 19
Diffusion Language Models: An Experimental Analysis
arXiv:2606. 19475v1 Announce Type: new Abstract: Large Language Models (LLMs) have revolutionized language modeling through autoregressive generation, enabling strong performance across a wide range of tasks.
arXiv AI
Jun 2
FLARE: Diffusion for Hybrid Language Model
arXiv:2606. 01774v1 Announce Type: cross Abstract: Autoregressive (AR) large language models (LLMs) have achieved broad practical success, but sequential decoding remains a key bottleneck for low-latency deployment.
arXiv Machine Learning
Jun 5
A Survey on Diffusion Language Models
arXiv:2508. 10875v3 Announce Type: replace-cross Abstract: Diffusion Language Models (DLMs) are rapidly emerging as a powerful and promising alternative to the dominant autoregressive (AR) paradigm.
arXiv Machine Learning
Aug 6
Esoteric Language Models: A Family of Any-Order Diffusion LLMs
arXiv:2506. 01928v5 Announce Type: replace-cross Abstract: Diffusion-based language models offer a compelling alternative to autoregressive (AR) models by enabling parallel and controllable generation.
arXiv Machine Learning
Jun 26
Dynamic-dLLM: Dynamic Cache-Budget and Adaptive Parallel Decoding for Training-Free Acceleration of Diffusion LLM
arXiv:2606. 26120v1 Announce Type: cross Abstract: Diffusion Large Language Models (dLLMs) offer a promising alternative to autoregressive models, excelling in text generation tasks due to their bidirectional attention mechanisms.
arXiv Machine Learning
Aug 10
Retrofitting Linear Attention into Diffusion Language Models
arXiv:2608. 06628v1 Announce Type: new Abstract: Diffusion language models (dLLMs) offer a promising alternative to autoregressive models by accelerating inference through parallel decoding.
Hugging Face Trending Papers
Jun 25
Nemotron-TwoTower: Diffusion Language Modeling with Pretrained Autoregressive Context
Diffusion language models offer a promising alternative to autoregressive models due to their potential for parallel and iterative generation. However, existing approaches use a single network for both context representation and iterative denoising, forcing one model to serve both roles and limiting its capacity for either role.
arXiv Machine Learning
Aug 4
A Comparative analysis of Layer-wise Representational Capacity in AR and Diffusion LLMs
arXiv:2603. 07475v4 Announce Type: replace-cross Abstract: Autoregressive (AR) language models build representations incrementally via left-to-right prediction, while diffusion language models (dLLMs) are trained through full-sequence denoising.
arXiv AI
Aug 7
d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillation
arXiv:2601. 07568v3 Announce Type: replace-cross Abstract: Diffusion large language models (dLLMs) offer capabilities beyond those of autoregressive (AR) LLMs, such as parallel decoding and random-order generation.
arXiv AI
Jul 29
PreDiff-LM: Pretrained Discrete Masked Diffusion Language Modeling with Hybrid Attention
arXiv:2607. 25157v1 Announce Type: new Abstract: Discrete masked diffusion language models support bidirectional generation and infilling, but adapting pretrained autoregressive (AR) transformers requires reconciling causal pretraining with bidirectional denoising.
arXiv AI
Jul 15
Accelerating Masked Diffusion Large Language Models: A Survey of Efficient Inference Techniques
arXiv:2607. 12829v1 Announce Type: cross Abstract: Diffusion large language models (dLLMs) offer a theoretical advantage in parallel generation over standard autoregressive models.