arXiv:2606. 04535v1 Announce Type: cross Abstract: Diffusion large language models (dLLMs) offer bidirectional attention and parallel generation, enabling them to exploit global context and naturally support format-constrained tasks like parseable JSON or reasoning templates.
arXiv:2601. 12247v3 Announce Type: replace-cross Abstract: Diffusion Language Models (DLMs) present a promising non-sequential paradigm for text generation, distinct from standard autoregressive (AR) approaches.
The paper introduces Causal Shortcut Learning (CSL), a framework that identifies token chains—called causal shortcuts—that guide Diffusion Language Models (DLMs) toward correct reasoning paths. By extracting these shortcuts and applying parallel prioritized masking during training, CSL improves both convergence speed and generation accuracy. Experiments on several reasoning benchmarks and two base models show CSL outperforms existing SFT-variant baselines, achieving an average 1.92% improvement over SFT-only models and up to 4.20% on MATH-500.
Sep‑Latent introduces separator tokens as natural compression anchors for implicit reasoning in large language models, enabling the model to aggregate and compress reasoning logic into these anchors via an Attention‑Decay mechanism. The framework aligns explicit and implicit pathways through multi‑level distillation, ensuring logical consistency, and allows dynamic reasoning depth by terminating the latent thinking phase based on query complexity. On the GSM8k‑Aug benchmark, Sep‑Latent achieves 56.8% accuracy with an average of only 5.4 tokens, outperforming prior fixed‑length implicit models in efficiency while narrowing the gap with explicit Chain‑of‑Thought methods.
The paper introduces PILL, a new infilling technique for diffusion language models that eliminates the need for a preset initial length and reduces inference overhead. PILL uses probing-based length-free decoding, cutting down on extra forward passes and speeding up generation. Experiments across five diffusion models and eight benchmarks show PILL outperforms the strongest baseline with higher pass rates and BLEU-2 scores while running 1.82× faster.
arXiv:2606. 19475v1 Announce Type: new Abstract: Large Language Models (LLMs) have revolutionized language modeling through autoregressive generation, enabling strong performance across a wide range of tasks.
arXiv:2602. 11133v2 Announce Type: replace Abstract: Diffusion language models generate text through iterative refinement, a process that is often computationally inefficient because many tokens reach stability long before the final denoising step.
arXiv:2508. 10875v3 Announce Type: replace-cross Abstract: Diffusion Language Models (DLMs) are rapidly emerging as a powerful and promising alternative to the dominant autoregressive (AR) paradigm.
The paper introduces Less Uniform Diffusion (LUDI), a framework that improves uniform diffusion language models by using a less uniform loss and per-token time embeddings to guide reverse transitions and enable confidence-based few-step sampling. Experiments demonstrate that LUDI provides cleaner supervision, enhances few-step generation, and scales to a 7B model (LUDI-7B) that achieves a 3-token-per-step speedup over autoregressive decoding while matching masked diffusion baselines. The work suggests that UDLMs still have untapped potential for complex generation tasks.
The paper introduces Seq-Drifting, a new framework that enables diffusion language models to generate an entire text sequence in a single forward pass. By applying a drift field to continuous token representations, the model learns a direct mapping from Gaussian noise to text, using attraction to guide outputs toward contextually plausible tokens and repulsion to prevent repetition and collapse. Experiments show that Seq-Drifting achieves a perplexity of 36.01 on LM1B and performs competitively in unconditional, conditional, translation, summarization, and reasoning tasks.
arXiv:2604. 08571v3 Announce Type: replace-cross Abstract: While Large Language Models (LLMs) achieve high performance on standard mathematical benchmarks, their problem-solving abilities depend on the context and textual formatting.
arXiv:2609.07160v1 Announce Type: cross Abstract: Diffusion Large Language Models (dLLMs) generate text via bidirectional iterative denoising, naturally supporting user-specified constraints anchored...
The paper introduces Stitching Noisy Diffusion Thoughts, a self‑consistency framework that transforms low‑cost diffusion‑sampled reasoning into a reusable pool of step‑level candidates. By sampling many diverse trajectories with a masked diffusion language model, scoring each intermediate step with a process reward model, and stitching the highest‑quality steps across trajectories, the method recomputes only the final answer, separating exploration from evaluation. Experiments on math reasoning benchmarks show that step‑level recombination improves accuracy by up to 23.8% on six tasks while reducing latency by up to 1.8× compared to traditional diffusion models and unified architectures.