The Flow has not summarised this story yet — read it at arXiv Computation and Language.
RangeSWA is a training‑free method designed to improve length extrapolation in continuous diffusion language models. It combines local sliding‑window attention, remote block summaries, and dynamic content retrieval to maintain local detail while accessing long‑range context. The approach also includes positional remapping, conditioning attention correction, and denoising‑stage‑adaptive window scheduling to mitigate generation degradation, resulting in lower perplexity and fewer repetitions in long‑text generation.
The paper introduces a survival-guided length control method for diffusion language models (DLMs), framing length selection as a discrete-time survival problem over the end-of-sequence token. This training‑free, plug‑in length predictor can be added to any existing DLM and reduces unnecessary denoising steps. Experiments on reasoning and code‑generation benchmarks show up to a seven‑fold speedup in inference while maintaining task accuracy, and reveal that predicted lengths vary significantly even within the same dataset, affecting model performance.
arXiv:2606. 08408v1 Announce Type: cross Abstract: We extend activation steering to diffusion language models (DLMs) and study a novel problem that arose due to the inference mechanism of DLMs: Modifying a text in-place to manifest a different concept.
The paper introduces Adjacent Possible Exploration (APE), a selective fine‑tuning strategy for large language models that evaluates multiple parameter updates on small data subsets and accepts only those that surpass a performance threshold. APE draws on evolutionary optimization ideas to filter out destabilizing changes, contrasting with conventional single‑gradient fine‑tuning. Experiments on news summarization show a 33.9% BLEU gain and a 36.2% perplexity drop while keeping computational demands low.
arXiv:2605.28456v2 Announce Type: replace Abstract: Existing Visual Speech Recognition (VSR) systems commonly rely on left-to-right autoregressive decoding, which can force premature decisions on vis...
BiMoGen introduces a unified masked discrete diffusion framework for bidirectional motion‑text generation, addressing limitations of autoregressive models by allowing iterative bidirectional prediction. The method employs a two‑stage training strategy—Decoupled Uni‑ and Cross‑Modal Training followed by supervised fine‑tuning—to establish cross‑modal correspondence and specialize for bidirectional generation. To mitigate train‑test mismatch, Generation‑Aware Self‑Correction exposes the model to its own predictions during training and applies correction passes early in sampling to refine unmasked tokens, achieving competitive results on HumanML3D and KIT‑ML.