arXiv Machine Learning

Formalizing the Sampling Design Space of Diffusion-Based Generative Models via Adaptive Solvers and Wasserstein-Bounded Timesteps

arXiv Machine Learning
Oct 5

Adaptive Second-Order Solvers for Fast Stochastic Diffusion Sampling

The paper introduces adaptive second‑order solvers for diffusion models, employing a proportional‑integral (PI) step‑size controller based on a diffusion noise‑normalised error estimator. Unlike existing adaptive methods that react only to the current error, the PI solver incorporates past error information, producing smoother step adaptations. The authors also demonstrate that per‑sample trajectories share structure, allowing aggregation into a fixed schedule that largely preserves adaptive benefits. Experiments on natural‑image and language datasets show that the fixed schedule outperforms the commonly used EDM schedule for image sampling with the stochastic Heun sampler, and that the PI adaptive solver achieves lower FID than most baselines, though it does not surpass the EDM‑churn sampler at low neural‑network‑evaluation counts. In language diffusion, the PI solver improves perplexity over EDM and entropy schedules at low‑to‑medium NFE, albeit with reduced token entropy. The study concludes that per‑sample adaptivity is highly beneficial for simple 1D toy problems but only marginally useful for image and language data, where an average schedule can sometimes outperform the PI‑adaptive approach.

By Ella Kemperman, Luca Ambrogioni
arXiv Machine Learning
Oct 16

FlowBudget: Training-Free Sampling-Budget Estimation for Flow Matching Models

FlowBudget introduces a training‑free method to estimate the sampling budget—i.e., the number of function evaluations (NFEs)—required for flow‑matching models. By using simulation‑free interpolation probes to normalize one‑step discretization error, it derives estimators for common solvers (Euler, Heun, Euler–Maruyama) that map a numerical tolerance to a step budget. Experiments on several flow‑model architectures demonstrate that FlowBudget can predict the necessary NFE without generating samples at multiple budgets or conducting expensive FID‑versus‑NFE sweeps.

By Zahra Dehghani, Pablo Piantanida, Mohammadhadi Shateri
arXiv AI
Jun 2

Efficient Weighted Sampling via Score-based Generative Models

arXiv:2502. 04646v2 Announce Type: replace-cross Abstract: Weighted sampling -- sampling from a probability density function (PDF) proportional to the product of a base PDF and a weight function -- is a fundamental technique with wide-ranging applications in variance reduction, biased sampling, data augmentation, and more.

By Heasung Kim, Taekyun Lee, Hyeji Kim, Gustavo de Veciana
arXiv AI
3d ago

ART for Diffusion Sampling: A Reinforcement Learning Approach to Timestep Scheduling

The paper introduces Adaptive Reparameterized Time (ART), a method that adjusts the clock speed of a reparameterized time variable to redistribute computation across a diffusion sampling trajectory, aiming to minimize Euler discretization error. A randomized companion, ART-RL, reformulates ART as a continuous-time reinforcement learning problem with Gaussian policies, establishing a two-directional bridge between deterministic ART optima and optimal Gaussian policies. Experiments show that ART-RL improves FID scores on CIFAR‑10 across various budgets, and the distilled deterministic schedule can transfer to other datasets without additional inference cost.

By Yilie Huang, Wenpin Tang, Xun Yu Zhou
arXiv AI
Sep 3

GeoSPRINT: Geometric Redundancy-Aware Step Pruning for Inference in Diffusion Trajectories

GeoSPRINT is a training‑free framework that constructs non‑uniform sampling schedules for diffusion model inference by detecting geometrically redundant steps in denoising trajectories. It uses a hyperplanarity test in latent space, implemented via QR factorization, to allocate more steps to high‑curvature regions, and introduces the trajectory projection score α_traj as a model‑free diagnostic for flow quality. Across CIFAR‑10, LSUN Church, and Stable Diffusion v1.5, GeoSPRINT consistently outperforms uniform DDIM schedules at matched NFE budgets, improving FID scores by up to 1.93 points.

By Arpita Joshi