The paper introduces adaptive second‑order solvers for diffusion models, employing a proportional‑integral (PI) step‑size controller based on a diffusion noise‑normalised error estimator. Unlike existing adaptive methods that react only to the current error, the PI solver incorporates past error information, producing smoother step adaptations. The authors also demonstrate that per‑sample trajectories share structure, allowing aggregation into a fixed schedule that largely preserves adaptive benefits. Experiments on natural‑image and language datasets show that the fixed schedule outperforms the commonly used EDM schedule for image sampling with the stochastic Heun sampler, and that the PI adaptive solver achieves lower FID than most baselines, though it does not surpass the EDM‑churn sampler at low neural‑network‑evaluation counts. In language diffusion, the PI solver improves perplexity over EDM and entropy schedules at low‑to‑medium NFE, albeit with reduced token entropy. The study concludes that per‑sample adaptivity is highly beneficial for simple 1D toy problems but only marginally useful for image and language data, where an average schedule can sometimes outperform the PI‑adaptive approach.
By Ella Kemperman, Luca Ambrogioni
arXiv:2602.12624v2 Announce Type: replace
Abstract: Diffusion-based generative models have achieved remarkable performance across various domains, yet their practical deployment is often limited by h...
By Sangwoo Jo, Sungjoon Choi
arXiv:2607.02137v3 Announce Type: replace-cross
Abstract: We study timestep allocation for score-based diffusion sampling, where a learned reverse-time dynamics is discretized on a finite grid. Unifo...
By Yilie Huang, Wenpin Tang, Xun Yu Zhou
arXiv:2607. 20540v1 Announce Type: cross Abstract: How should a diffusion model decide which noise levels to train on, and how much?
By Luca Ambrogioni, Giulio Franzese, Alberto Foresti, Gabriel Raya, Bac Nguyen, Georgios Batzolis, Yuhta Takida, Naoki Murata, Chieh-Hsin Lai, Yuki Mitsufuji
GeoSPRINT is a training‑free framework that constructs non‑uniform sampling schedules for diffusion model inference by detecting geometrically redundant steps in denoising trajectories. It uses a hyperplanarity test in latent space, implemented via QR factorization, to allocate more steps to high‑curvature regions, and introduces the trajectory projection score α_traj as a model‑free diagnostic for flow quality. Across CIFAR‑10, LSUN Church, and Stable Diffusion v1.5, GeoSPRINT consistently outperforms uniform DDIM schedules at matched NFE budgets, improving FID scores by up to 1.93 points.
By Arpita Joshi
Forward-Free Diffusion Language Models with BPTT-Free Looped Refinement (FReDA) removes the need for a hand‑designed forward process in diffusion language modeling by treating model‑generated drafts as implicit intermediate states and refining them recursively. The approach detaches earlier refinement passes, backpropagating only through the final pass, and supports both self‑refinement and Best‑of‑N candidate selection. In sub‑8B experiments, FReDA‑4B surpasses larger diffusion baselines on reasoning and coding tasks, achieving up to 15% absolute gains and a 1.5‑1.8× speedup while scaling well with additional refinement steps.
By Haotian Sun, Rushi Qiang, Yuqian Zheng, Bo Dai
arXiv:2605. 00161v2 Announce Type: replace Abstract: Diffusion language models (DLMs) are an attractive alternative to autoregressive models because they promise sublinear-time, parallel generation, yet practical gains remain elusive as high-quality samples still demand hundreds of refinement steps.
By Hasan Amin, Yuan Gao, Yaser Souri, Subhojit Som, Ming Yin, Rajiv Khanna, Xia Song
The paper studies how to allocate a fixed computational budget across the denoising steps of diffusion models to improve sample quality at deployment. It shows that the expected benefit of evaluating multiple candidates at a step can be decomposed into a step‑specific sensitivity and a universal sample‑size factor, and that the optimal allocation follows a water‑filling structure. Experiments demonstrate that this allocation achieves the same quality as a uniform strategy while reducing function evaluations by 20–50%.
By Yuan Cao, Yifu Tang, Hangqi Li, Zeyu Zheng
arXiv:2507. 08390v5 Announce Type: replace Abstract: Discrete diffusion models have recently emerged as strong alternatives to autoregressive language models, matching their performance through large-scale training.
By Meihua Dang, Jiaqi Han, Minkai Xu, Kai Xu, Akash Srivastava, Stefano Ermon
arXiv:2602. 15008v2 Announce Type: replace Abstract: Diffusion models over discrete spaces have recently shown striking empirical success, yet their theoretical foundations remain incomplete.
By Daniil Dmitriev, Zhihan Huang, Yuting Wei
arXiv:2607. 26285v1 Announce Type: cross Abstract: Two central challenges in diffusion-based sampling are the theoretical one of understanding their remarkable effectiveness even in high-dimensional settings, and the practical one of designing algorithms with certified performance guarantees.
By Martin J. Wainwright
We study timestep allocation for score-based diffusion sampling, where a learned reverse-time dynamics is discretized on a finite grid. Uniform and hand-crafted schedules are standard choices, but they rely on fixed prescriptions and can therefore be suboptimal.