arXiv Computation and Language August 28, 2026 By Haotian Sun, Rushi Qiang, Yuqian Zheng, Bo Dai

Forward-Free Diffusion Language Models with BPTT-Free Looped Refinement

Read the original on arXiv Computation and Language →

Forward-Free Diffusion Language Models with BPTT-Free Looped Refinement (FReDA) removes the need for a hand‑designed forward process in diffusion language modeling by treating model‑generated drafts as implicit intermediate states and refining them recursively. The approach detaches earlier refinement passes, backpropagating only through the final pass, and supports both self‑refinement and Best‑of‑N candidate selection. In sub‑8B experiments, FReDA‑4B surpasses larger diffusion baselines on reasoning and coding tasks, achieving up to 15% absolute gains and a 1.5‑1.8× speedup while scaling well with additional refinement steps.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Hugging Face Trending Papers

Oct 6

Enhancing Diffusion Language Models with Autoregressive Post-Training Weights

Diffusion language models (dLLMs) have emerged as a promising alternative to autoregressive (AR) language models, offering flexible token-update orders and parallel decoding. Recent dLLMs are often in...

llms diffusion reinforcement-learning fine-tuning

arXiv Machine Learning

Oct 7

Enhancing Diffusion Language Models with Autoregressive Post-Training Weights

arXiv:2610.08108v1 Announce Type: new Abstract: Diffusion language models (dLLMs) have emerged as a promising alternative to autoregressive (AR) language models, offering flexible token-update orders...

By Yiming Qin, Ke Wang, Amel Abdelraheem, Adam Hazimeh, Pascal Frossard

llms diffusion reinforcement-learning fine-tuning

arXiv Machine Learning

Sep 7

Distilled Continuous Diffusion Language Models Can Write Code in Few Steps---or One

PlaidQ is a 0.7B continuous diffusion language model designed for code generation. By distilling its iterative refinement trajectory into only a few denoising steps—or even a single step—PlaidQ achieves competitive performance with discrete diffusion models while dramatically reducing inference time. The study demonstrates that continuous diffusion can be effectively compressed, enabling efficient and accurate code generation with minimal computational overhead.

By Fred Zhangzhi Peng, Kaiwen Zheng, Anru R. Zhang

llms rag diffusion efficiency benchmarks

arXiv Machine Learning

Jun 2

Consistent Diffusion Language Models

arXiv:2605. 00161v2 Announce Type: replace Abstract: Diffusion language models (DLMs) are an attractive alternative to autoregressive models because they promise sublinear-time, parallel generation, yet practical gains remain elusive as high-quality samples still demand hundreds of refinement steps.

By Hasan Amin, Yuan Gao, Yaser Souri, Subhojit Som, Ming Yin, Rajiv Khanna, Xia Song

llms diffusion efficiency

arXiv Machine Learning

Jun 11

Teaching Diffusion to Speculate Left-to-Right

arXiv:2606. 11552v1 Announce Type: cross Abstract: Large language models (LLMs) achieve remarkable performance across a wide range of tasks, but their autoregressive decoding process incurs substantial inference costs due to inherently sequential token generation.

By Lexington Whalen, Yuki Ito, Ryo Sakamoto

llms diffusion benchmarks safety

arXiv Machine Learning

Jul 22

AdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafters

arXiv:2607. 19223v1 Announce Type: new Abstract: Speculative decoding, in which a lightweight draft model first generates a draft sequence that is then verified in parallel by the target model, has become a prevalent paradigm for accelerating large language model inference.

By Yu-Yang Qian, Hao-Cong Wu, Chen Chen, Jiacheng Sun, Zhenhua Dong, Peng Zhao, Zhi-Hua Zhou

llms diffusion efficiency benchmarks

One email a morning, machine-written

Related stories

Enhancing Diffusion Language Models with Autoregressive Post-Training Weights

Enhancing Diffusion Language Models with Autoregressive Post-Training Weights

Distilled Continuous Diffusion Language Models Can Write Code in Few Steps---or One

Consistent Diffusion Language Models

Teaching Diffusion to Speculate Left-to-Right

AdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafters