arXiv Computation and Language August 28, 2026 By Haotian Sun, Rushi Qiang, Yuqian Zheng, Bo Dai

Forward-Free Diffusion Language Models with BPTT-Free Looped Refinement

Read the original on arXiv Computation and Language →

Forward-Free Diffusion Language Models with BPTT-Free Looped Refinement (FReDA) removes the need for a hand‑designed forward process in diffusion language modeling by treating model‑generated drafts as implicit intermediate states and refining them recursively. The approach detaches earlier refinement passes, backpropagating only through the final pass, and supports both self‑refinement and Best‑of‑N candidate selection. In sub‑8B experiments, FReDA‑4B surpasses larger diffusion baselines on reasoning and coding tasks, achieving up to 15% absolute gains and a 1.5‑1.8× speedup while scaling well with additional refinement steps.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Machine Learning

1d ago

Distilled Continuous Diffusion Language Models Can Write Code in Few Steps---or One

arXiv:2609.04531v1 Announce Type: new Abstract: Language generation is almost universally treated as a sequential process: autoregressive models emit one token at a time, while diffusion language mod...

By Fred Zhangzhi Peng, Kaiwen Zheng, Anru R. Zhang

llms rag diffusion efficiency benchmarks

arXiv Machine Learning

Jun 2

Consistent Diffusion Language Models

arXiv:2605. 00161v2 Announce Type: replace Abstract: Diffusion language models (DLMs) are an attractive alternative to autoregressive models because they promise sublinear-time, parallel generation, yet practical gains remain elusive as high-quality samples still demand hundreds of refinement steps.

By Hasan Amin, Yuan Gao, Yaser Souri, Subhojit Som, Ming Yin, Rajiv Khanna, Xia Song

llms diffusion efficiency

arXiv Machine Learning

Jun 11

Teaching Diffusion to Speculate Left-to-Right

arXiv:2606. 11552v1 Announce Type: cross Abstract: Large language models (LLMs) achieve remarkable performance across a wide range of tasks, but their autoregressive decoding process incurs substantial inference costs due to inherently sequential token generation.

By Lexington Whalen, Yuki Ito, Ryo Sakamoto

llms diffusion benchmarks safety

arXiv Machine Learning

Jul 22

AdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafters

arXiv:2607. 19223v1 Announce Type: new Abstract: Speculative decoding, in which a lightweight draft model first generates a draft sequence that is then verified in parallel by the target model, has become a prevalent paradigm for accelerating large language model inference.

By Yu-Yang Qian, Hao-Cong Wu, Chen Chen, Jiacheng Sun, Zhenhua Dong, Peng Zhao, Zhi-Hua Zhou

llms diffusion efficiency benchmarks

Hugging Face Trending Papers

Jul 21

AdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafters

Speculative decoding, in which a lightweight draft model first generates a draft sequence that is then verified in parallel by the target model, has become a prevalent paradigm for accelerating large language model inference. Recent work such as DFlash further boosts drafting efficiency by leveraging diffusion drafters, whose parallel denoising mechanism enables draft generation in a single forward pass.

llms diffusion efficiency benchmarks

arXiv AI

Jun 19

Diffusion Language Models: An Experimental Analysis

arXiv:2606. 19475v1 Announce Type: new Abstract: Large Language Models (LLMs) have revolutionized language modeling through autoregressive generation, enabling strong performance across a wide range of tasks.

By Thomas Bertolani, Davide Bucciarelli, Leonardo Zini, Marcella Cornia, Lorenzo Baraldi

llms diffusion benchmarks

One email a morning, machine-written

Related stories

Distilled Continuous Diffusion Language Models Can Write Code in Few Steps---or One

Consistent Diffusion Language Models

Teaching Diffusion to Speculate Left-to-Right

AdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafters

AdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafters

Diffusion Language Models: An Experimental Analysis