Hugging Face Trending Papers September 28, 2026

Low-Confidence Remasking Traps Flexibility: Realizing Arbitrary-Order Potential for Diverse Rollouts in Diffusion LLMs

Read the original on Hugging Face Trending Papers →

Masked diffusion language models can generate outputs in any order, but recent findings suggest this flexibility may reduce diversity by delaying uncertain tokens. The study identifies low‑confidence remasking (LCR) as the main culprit, showing that its filtering of lower‑probability tokens suppresses diversity exponentially. Replacing LCR with top‑probability position selection (TPP) restores diversity, and adding Entropy‑Guided Initialization (EGI) further enhances rollout diversity and solution coverage, demonstrating the benefits of arbitrary‑order generation for diverse outputs.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Hugging Face Trending Papers.

Hugging Face Trending Papers

Jun 10

Re-evaluating Confidence Remasking in Masked Diffusion Language Models

Masked diffusion language models (dLLMs) have recently emerged as a competitive alternative to autoregressive language models, with the promise of faster inference via parallel token generation. A notable limitation of the masked formulation, however, is that once a token has been unmasked it can no longer be revised, leaving dLLMs vulnerable to early sampling mistakes.

llms diffusion

arXiv Machine Learning

Jun 11

Re-evaluating Confidence Remasking in Masked Diffusion Language Models

arXiv:2606. 12232v1 Announce Type: new Abstract: Masked diffusion language models (dLLMs) have recently emerged as a competitive alternative to autoregressive language models, with the promise of faster inference via parallel token generation.

By Stipe Frkovic, Metod Jazbec, Dan Zhang, Christian A. Naesseth, Ilija Bogunovic, Eric Nalisnick

llms diffusion

arXiv Machine Learning

Jun 16

Mean-Field Parallel Decoding for Discrete Diffusion Language Models

arXiv:2606. 15805v1 Announce Type: new Abstract: Discrete diffusion language models enable parallel token generation, offering a pathway to low-latency decoding.

By Tamim Zoabi, Ameen Ali, Liran Ringel, Lior Wolf

llms diffusion benchmarks

Hugging Face Trending Papers

Jul 27

Rethinking the Generation Order of Block Diffusion Language Models

Diffusion language models enable flexible arbitrary-order generation, but existing sampling methods are mostly designed for early masked diffusion models (MDMs). In this work, we study sampling for recent block diffusion language models (BDLMs).

llms diffusion

arXiv Machine Learning

2d ago

Self-Repulsive Sampling for Diffusion Language Models

arXiv:2609.39560v1 Announce Type: new Abstract: Sampling several responses and voting over their answers can improve a language model's accuracy, but repeated answers limit the benefit of additional...

By Michael Helcig, Martin Jaggi

llms diffusion benchmarks

arXiv Machine Learning

Jul 8

Diffusion Language Model Parallel Decoding via Product-of-Experts Bridge

arXiv:2606. 08048v1 Announce Type: cross Abstract: Diffusion language models (DLMs) offer substantial speed advantages through parallel decoding, but the lack of token dependencies limits generation quality compared to autoregressive (AR) models.

By Juntong Shi, Brian L. Trippe, Jure Leskovec, Stefano Ermon, Minkai Xu

llms diffusion

One email a morning, machine-written

Related stories

Re-evaluating Confidence Remasking in Masked Diffusion Language Models

Re-evaluating Confidence Remasking in Masked Diffusion Language Models

Mean-Field Parallel Decoding for Discrete Diffusion Language Models

Rethinking the Generation Order of Block Diffusion Language Models

Self-Repulsive Sampling for Diffusion Language Models

Diffusion Language Model Parallel Decoding via Product-of-Experts Bridge