← Back to all news
arXiv Machine Learning By Vansh Bansal, Cholyeon Cho, Syamantak Kumar, Sujay Sanghavi, Purnamrita Sarkar

Walk fast but be careful: Understanding Parallel Sampling in Masked Diffusion

Read the original on arXiv Machine Learning →

The paper introduces random walks on graphs as a controllable sandbox for studying parallel sampling strategies in masked diffusion models (MDMs). It shows that common parallel unmasking methods, such as lowest‑entropy scoring, are not universally superior to random sampling and that performance depends on the graph’s conditional dependence structure. The authors also propose training‑free bisection samplers that are provably exact for random walks and demonstrate their effectiveness on both graph‑walk tasks and pretrained MDMs for language generation and reasoning benchmarks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Jun 29

Masked Language Flow Models

arXiv:2606. 27617v1 Announce Type: cross Abstract: Masked Diffusion Models (MDMs) promise fast, parallel language generation, but their reverse transition factorises across token positions -- an approximation that breaks down in the few-step sampling regime where parallel generation ought to provide the greatest efficiency gains.

By Iskander Azangulov, Kianoosh Ashouritaklimi, Leo Zhang, Simon Vary, Patrick Rebeschini
llmsdiffusion
More like this →
arXiv AI
5d ago

Test-Time Scaling with Diffusion Language Models via Reward-Guided Stitching

The paper introduces Stitching Noisy Diffusion Thoughts, a self‑consistency framework that transforms low‑cost diffusion‑sampled reasoning into a reusable pool of step‑level candidates. By sampling many diverse trajectories with a masked diffusion language model, scoring each intermediate step with a process reward model, and stitching the highest‑quality steps across trajectories, the method recomputes only the final answer, separating exploration from evaluation. Experiments on math reasoning benchmarks show that step‑level recombination improves accuracy by up to 23.8% on six tasks while reducing latency by up to 1.8× compared to traditional diffusion models and unified architectures.

By Roy Miles, Aysim Toker, Andreea-Maria Oncescu, Jiankang Deng, Ismail Elezi
llmsdiffusionreinforcement-learningbenchmarks
More like this →
arXiv AI
Sep 30

Less Uniform Discrete Diffusion is More Powerful and Scalable

The paper introduces Less Uniform Diffusion (LUDI), a framework that improves uniform diffusion language models by using a less uniform loss and per-token time embeddings to guide reverse transitions and enable confidence-based few-step sampling. Experiments demonstrate that LUDI provides cleaner supervision, enhances few-step generation, and scales to a 7B model (LUDI-7B) that achieves a 3-token-per-step speedup over autoregressive decoding while matching masked diffusion baselines. The work suggests that UDLMs still have untapped potential for complex generation tasks.

By Kaibo Wang, Ding Ding, Fangyu Ding, Zijin Feng, Han Shi, Haili Bai, Jiacheng Sun, Yang Xiang
llmsragdiffusion
More like this →