← Back to all news
arXiv Machine Learning By Yuki Ichihara, Naoto Iwase, Mohammad Atif Quamar, Junpei Komiyama

Prefix-Denoising Consistency: Test-Time Verification for Diffusion Language Models

Read the original on arXiv Machine Learning →

The paper introduces Prefix-Denoising Consistency (PDC), a test‑time self‑verification technique for Diffusion Language Models (DLMs). PDC leverages the stability of correct output trajectories by splitting a generated sentence at an intermediate point and regenerating the remainder conditioned on the fixed prefix. Experiments on mathematical and commonsense reasoning benchmarks show that PDC consistently improves initial samples, outperforms independent generations under computational constraints, and remains robust across different unmasking strategies and parameter settings.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Jun 15

Residual Context Diffusion Language Models

arXiv:2601. 22954v2 Announce Type: replace-cross Abstract: Diffusion Large Language Models (dLLMs) have emerged as a promising alternative to purely autoregressive language models because they can decode multiple tokens in parallel.

By Yuezhou Hu, Harman Singh, Monishwaran Maheswaran, Haocheng Xi, Coleman Hooper, Jintao Zhang, Aditya Tomar, Michael W. Mahoney, Sewon Min, Mehrdad Farajtabar, Kurt Keutzer, Amir Gholami, Chenfeng Xu
llmsdiffusionbenchmarks
More like this →
arXiv Computation and Language
Aug 28

Dependency-Aware Revocable Decoding for Efficient Diffusion Large Language Model Inference

The paper introduces Dependency-Aware Revocable Decoding (DARD), a training‑free framework for diffusion large language models that separates tokens into masked, candidate, and unmasked states. DARD verifies candidate tokens using a selective context that excludes less reliable tokens and adaptively regulates their influence on subsequent decoding. Experiments on 12 textual and multimodal benchmarks across three open‑source dLLMs show that DARD improves the speed‑quality Pareto frontier, achieving a 2.71× speedup and a 4.35‑point CIDEr gain over Saber on Flickr30K.

By Wooje Park, Insu Lee, Minyoung Noh, Jaeyun Jang, Sungmin Lee, Kyuhong Shim, Byonghyo Shim
llmsdiffusionmultimodalbenchmarks
More like this →
arXiv AI
5d ago

Predict, Don't Iterate: Efficient Adaptive-Length Infilling for Diffusion Language Models

The paper introduces PILL, a new infilling technique for diffusion language models that eliminates the need for a preset initial length and reduces inference overhead. PILL uses probing-based length-free decoding, cutting down on extra forward passes and speeding up generation. Experiments across five diffusion models and eight benchmarks show PILL outperforms the strongest baseline with higher pass rates and BLEU-2 scores while running 1.82× faster.

By Haobo Xu, Sirui Chen, Yuanchen Bei, Lingjie Chen, Yuchen Yan, Dongqi Fu, Jingrui He, Hanghang Tong
llmsdiffusionbenchmarks
More like this →