arXiv:2606. 10829v1 Announce Type: cross Abstract: Masked diffusion language models can reduce inference steps by revealing multiple tokens per denoising iteration, but this parallelism is fragile: positions that are individually confident may be unsafe to commit together when their predictions are coupled.
By Yusuf Sahin, Ahmed Rockey Saikia, Volkan Cevher, Paolo Favaro
arXiv:2608.25505v2 Announce Type: replace-cross
Abstract: Masked diffusion models generate sequences in rounds, predicting several masked positions at once. Fewer rounds can accelerate decoding, but...
By Chuling Wen, Weijie Liang, Jian Lu, Zhouchen Lin
The paper introduces random walks on graphs as a controllable sandbox for studying parallel sampling strategies in masked diffusion models (MDMs). It shows that common parallel unmasking methods, such as lowest‑entropy scoring, are not universally superior to random sampling and that performance depends on the graph’s conditional dependence structure. The authors also propose training‑free bisection samplers that are provably exact for random walks and demonstrate their effectiveness on both graph‑walk tasks and pretrained MDMs for language generation and reasoning benchmarks.
By Vansh Bansal, Cholyeon Cho, Syamantak Kumar, Sujay Sanghavi, Purnamrita Sarkar
The paper introduces Fork-dLLM, a hybrid sampling strategy for diffusion language models that combines parallel token generation with autoregressive ordering only at uncertain fallback steps, thereby avoiding the flexibility trap that reduces generation diversity. It also presents ForkGRPO, an extension that applies the GRPO objective selectively during post‑training rollouts, preserving policy‑likelihood ratios while cutting rollout and optimization costs. Experiments show Fork-dLLM achieves pass@k scaling comparable to autoregressive sampling but 2–3× more efficient, and ForkGRPO matches or surpasses AR‑based GRPO baselines at significantly lower training expense.
By Stipe Frkovi\'c, Metod Jazbec, Christian A. Naesseth
arXiv:2607. 16207v1 Announce Type: new Abstract: Membership inference attacks (MIAs) test whether a candidate example appeared in a model's training data.
By Yeachan Jun, Albert No
Masked diffusion models (MDMs) generate discrete data by iteratively unmasking multiple tokens in parallel using a learned mask predictor. The paper provides end‑to‑end statistical guarantees for transformer‑based MDMs, showing that when the data distribution is well approximated by mixtures of hidden Markov trees, the model can achieve ε‑accurate sampling in KL divergence with ∫O(LK_par/ε) training samples, where K_par is polynomial in the sequence length, vocabulary size, number of hidden states, and mixture components. The transformer network and sampling procedure adapt automatically to the unknown latent structure without requiring prior knowledge of tree topologies or component parameters.
By Jingda Wu, Changxiao Cai