The paper introduces a survival-guided length control method for diffusion language models (DLMs), framing length selection as a discrete-time survival problem over the end-of-sequence token. This training‑free, plug‑in length predictor can be added to any existing DLM and reduces unnecessary denoising steps. Experiments on reasoning and code‑generation benchmarks show up to a seven‑fold speedup in inference while maintaining task accuracy, and reveal that predicted lengths vary significantly even within the same dataset, affecting model performance.
arXiv:2602. 11133v2 Announce Type: replace Abstract: Diffusion language models generate text through iterative refinement, a process that is often computationally inefficient because many tokens reach stability long before the final denoising step.
arXiv:2605.28456v2 Announce Type: replace Abstract: Existing Visual Speech Recognition (VSR) systems commonly rely on left-to-right autoregressive decoding, which can force premature decisions on vis...
arXiv:2606. 19475v1 Announce Type: new Abstract: Large Language Models (LLMs) have revolutionized language modeling through autoregressive generation, enabling strong performance across a wide range of tasks.
arXiv:2601. 17917v3 Announce Type: replace Abstract: Diffusion Large Language Models (dLLMs) offer a compelling paradigm for natural language generation, leveraging parallel decoding and bidirectional attention to achieve superior global coherence compared to autoregressive models.
arXiv:2611.06371v1 Announce Type: new Abstract: Summarization with a fixed character length is important for tasks such as headline and article preview generation, where text must fit an exact space....
The paper introduces PILL, a new infilling technique for diffusion language models that eliminates the need for a preset initial length and reduces inference overhead. PILL uses probing-based length-free decoding, cutting down on extra forward passes and speeding up generation. Experiments across five diffusion models and eight benchmarks show PILL outperforms the strongest baseline with higher pass rates and BLEU-2 scores while running 1.82× faster.
arXiv:2508. 10875v3 Announce Type: replace-cross Abstract: Diffusion Language Models (DLMs) are rapidly emerging as a powerful and promising alternative to the dominant autoregressive (AR) paradigm.
RangeSWA is a training‑free method designed to improve length extrapolation in continuous diffusion language models. It combines local sliding‑window attention, remote block summaries, and dynamic content retrieval to maintain local detail while accessing long‑range context. The approach also includes positional remapping, conditioning attention correction, and denoising‑stage‑adaptive window scheduling to mitigate generation degradation, resulting in lower perplexity and fewer repetitions in long‑text generation.
arXiv:2609.16372v1 Announce Type: new Abstract: Masked diffusion language models (dLLMs) generate text by iteratively denoising masked tokens with bidirectional attention. Extending reasoning across...
arXiv:2606. 04236v1 Announce Type: cross Abstract: Discrete diffusion language models can generate text efficiently by updating multiple masked positions in parallel, but this parallelism introduces a quality-latency trade-off.
Diffusion large language models (dLLMs) re-encode the entire prefix at every denoising step, causing recomputation that scales quadratically with context length and becomes prohibitive for long-context scenarios. We propose Prefilling-dLLM, a training-free prefill-decode disaggregation framework for dLLMs that partitions the prefix into N chunks, caches their KV representations once, and selects the top-K most relevant chunks with intra-chunk token sparsity for decoding, showing that sparse prefilling can outperform dense attention while reducing per-step complexity from quadratic in the full sequence length to quadratic only in the decode length.
The paper introduces a training‑free structured suffix modeling technique to accelerate Diffusion Language Models (DLMs). It partitions the suffix into local, middle, and tail regions, retaining varying numbers of tokens per region and incorporating previous decoding results into current token representations. Experiments on three DLMs show significant speedups—up to 72.81× in long‑sequence inference—while often improving performance, and the method is compatible with existing acceleration strategies.