arXiv Computation and Language November 2, 2026 By Guangfu Guo, Xiaoqian Lu, Linsey Pang, Weiran Yao, Haolin Chen, Long Cheng, Kunpeng Liu

RangeSWA: Training-Free Length Extrapolation for Continuous Diffusion Language Models

Read the original on arXiv Computation and Language →

RangeSWA is a training‑free method designed to improve length extrapolation in continuous diffusion language models. It combines local sliding‑window attention, remote block summaries, and dynamic content retrieval to maintain local detail while accessing long‑range context. The approach also includes positional remapping, conditioning attention correction, and denoising‑stage‑adaptive window scheduling to mitigate generation degradation, resulting in lower perplexity and fewer repetitions in long‑text generation.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Computation and Language

1d ago

Training-Free Character-Length Control in Summarization with Diffusion Language Models

arXiv:2611.06371v1 Announce Type: new Abstract: Summarization with a fixed character length is important for tasks such as headline and article preview generation, where text must fit an exact space....

By Steffen Castle, Nils Feldhus, Christopher Ebert, Leonhard Hennig, Sebastian M\"oller

llms diffusion nlp fine-tuning benchmarks

arXiv AI

Oct 12

One-Step Text Generation by Seq-Drifting

The paper introduces Seq-Drifting, a new framework that enables diffusion language models to generate an entire text sequence in a single forward pass. By applying a drift field to continuous token representations, the model learns a direct mapping from Gaussian noise to text, using attraction to guide outputs toward contextually plausible tokens and repulsion to prevent repetition and collapse. Experiments show that Seq-Drifting achieves a perplexity of 36.01 on LM1B and performs competitively in unconditional, conditional, translation, summarization, and reasoning tasks.

By Jiayu Liu, Ruizhi Zhao, Tianyun Ji, Yee Whye Teh, Zhenya Huang, Ning Miao

llms diffusion nlp

arXiv Machine Learning

Jun 5

A Survey on Diffusion Language Models

arXiv:2508. 10875v3 Announce Type: replace-cross Abstract: Diffusion Language Models (DLMs) are rapidly emerging as a powerful and promising alternative to the dominant autoregressive (AR) paradigm.

By Tianyi Li, Mingda Chen, Bowei Guo, Zhiqiang Shen

llms diffusion nlp efficiency multimodal benchmarks

arXiv AI

Sep 15

Lost-in-the-Middle in Long-Text Generation: Synthetic Dataset, Evaluation Framework, and Mitigation

arXiv:2503.06868v2 Announce Type: replace-cross Abstract: Existing long-text generation methods produce lengthy outputs from short inputs, leaving long-input-to-long-output generation underexplored....

By Junhao Zhang, Richong Zhang, Fanshuang Kong, Ziyang Miao, Yanhan Ye, Yaowei Zheng

llms rag benchmarks safety

arXiv AI

Aug 18

MLLM-Guided Semantic Correction for Text-to-Video Generation

arXiv:2608. 16513v1 Announce Type: cross Abstract: Recent advances in diffusion models and Transformer architectures have led to significant progress in text-to-video generation.

By Junhao Chen, Zheqi Lv, Keting Yin, Shengyu Zhang, Zhou Zhao, Feiyang Chen, Xinyu Duan, Baoxing Huai, Fei Wu

llms diffusion multimodal benchmarks safety

arXiv Computation and Language

Aug 28

Survival-Guided Length Control for Efficient Diffusion Language Models

The paper introduces a survival-guided length control method for diffusion language models (DLMs), framing length selection as a discrete-time survival problem over the end-of-sequence token. This training‑free, plug‑in length predictor can be added to any existing DLM and reduces unnecessary denoising steps. Experiments on reasoning and code‑generation benchmarks show up to a seven‑fold speedup in inference while maintaining task accuracy, and reveal that predicted lengths vary significantly even within the same dataset, affecting model performance.

By Ivan Kobyzev, Abbas Ghaddar, Yufei Cui

llms diffusion benchmarks

One email a morning, machine-written

Related stories

Training-Free Character-Length Control in Summarization with Diffusion Language Models

One-Step Text Generation by Seq-Drifting

A Survey on Diffusion Language Models

Lost-in-the-Middle in Long-Text Generation: Synthetic Dataset, Evaluation Framework, and Mitigation

MLLM-Guided Semantic Correction for Text-to-Video Generation

Survival-Guided Length Control for Efficient Diffusion Language Models