arXiv Machine Learning
Looped Uniform Diffusion Language Models
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Jun 15
Residual Context Diffusion Language Models
arXiv:2601. 22954v2 Announce Type: replace-cross Abstract: Diffusion Large Language Models (dLLMs) have emerged as a promising alternative to purely autoregressive language models because they can decode multiple tokens in parallel.
arXiv AI
Jul 24
Understanding and Accelerating the Training of Masked Diffusion Language Models
arXiv:2605. 13026v2 Announce Type: replace-cross Abstract: Masked diffusion models (MDMs) have emerged as a promising alternative to autoregressive models (ARMs) for language modeling.
arXiv AI
Aug 25
CAI-DLLM: Convergence Aware Inference for Diffusion Language Models
arXiv:2608.22646v1 Announce Type: new Abstract: Diffusion language models can generate many tokens in parallel, but they still require repeated denoising steps during inference. This makes generation...
arXiv AI
Sep 3
Predict, Don't Iterate: Efficient Adaptive-Length Infilling for Diffusion Language Models
The paper introduces PILL, a new infilling technique for diffusion language models that eliminates the need for a preset initial length and reduces inference overhead. PILL uses probing-based length-free decoding, cutting down on extra forward passes and speeding up generation. Experiments across five diffusion models and eight benchmarks show PILL outperforms the strongest baseline with higher pass rates and BLEU-2 scores while running 1.82× faster.
arXiv AI
Sep 30
On Trajectory-Aware Training for Masked Diffusion Language Models
arXiv:2609.37974v1 Announce Type: cross Abstract: Masked diffusion models (MDMs) generate text by unmasking several tokens per step, but they are trained and sampled under different conditions. The m...
arXiv AI
Oct 16
Designing Transformers for Continuous Diffusion Language Models
The paper introduces the Language Diffusion Transformer (LangDiT), a new transformer architecture tailored for continuous diffusion language models (CDLMs). By incorporating feature and query-key normalization, wider noise conditioning, likelihood-shortcut gradient routing, calibration, and confidence-gated training self-conditioning, LangDiT achieves lower generation perplexity than existing models across various CDLM formulations. Experiments on the OpenWebText dataset show state-of-the-art perplexity scores, demonstrating the architecture’s effectiveness while preserving the underlying model representations and training objectives.