arXiv Computer Vision

Analyzing and Improving Fast Sampling of Text-to-Image Diffusion Models

arXiv AI
Sep 16

Efficient Text-to-Image Generation: An Adaptive Step Schedule Controller for Diffusion Models

The paper introduces an adaptive step schedule controller for text‑to‑image diffusion models, allowing the number of denoising steps to vary based on the complexity of the input prompt. By mixing step schedules of different sizes and monitoring error discrepancies at each timestep, the method switches schedules to maintain image quality while reducing inference time. Experiments on COCO and DiffusionDB demonstrate that this approach achieves faster generation without sacrificing visual fidelity.

By Kuluhan Binici, Cihan Acar, Shivam Aggarwal, Siying Liu, Tulika Mitra
arXiv Machine Learning
Aug 19

Optimize Your Sampling: Tuned Diffusion Sampling with Bayesian Optimization

The paper introduces Optimizing Your Sampling (OYS), a method that treats diffusion model timestep selection as a black-box optimization problem and uses Bayesian optimization to directly improve the target metric. OYS outperforms default schedules and the Align Your Steps approach on text-to-image generation, inpainting, and other image tasks, as shown by both quantitative and human evaluations. It works without additional training, applies to distilled models, and enhances both simple and sophisticated samplers, achieving 89–94% of a 50‑step schedule’s quality with only 5 steps and a tenfold reduction in inference cost.

By Travis Zhang, Christian Belardi, Justin Lovelace, Jin Peng Zhou, Saebyeol Shin, Carla P. Gomes, Kilian Q. Weinberger
arXiv Machine Learning
Oct 5

Adaptive Second-Order Solvers for Fast Stochastic Diffusion Sampling

The paper introduces adaptive second‑order solvers for diffusion models, employing a proportional‑integral (PI) step‑size controller based on a diffusion noise‑normalised error estimator. Unlike existing adaptive methods that react only to the current error, the PI solver incorporates past error information, producing smoother step adaptations. The authors also demonstrate that per‑sample trajectories share structure, allowing aggregation into a fixed schedule that largely preserves adaptive benefits. Experiments on natural‑image and language datasets show that the fixed schedule outperforms the commonly used EDM schedule for image sampling with the stochastic Heun sampler, and that the PI adaptive solver achieves lower FID than most baselines, though it does not surpass the EDM‑churn sampler at low neural‑network‑evaluation counts. In language diffusion, the PI solver improves perplexity over EDM and entropy schedules at low‑to‑medium NFE, albeit with reduced token entropy. The study concludes that per‑sample adaptivity is highly beneficial for simple 1D toy problems but only marginally useful for image and language data, where an average schedule can sometimes outperform the PI‑adaptive approach.

By Ella Kemperman, Luca Ambrogioni
arXiv AI
Sep 3

GeoSPRINT: Geometric Redundancy-Aware Step Pruning for Inference in Diffusion Trajectories

GeoSPRINT is a training‑free framework that constructs non‑uniform sampling schedules for diffusion model inference by detecting geometrically redundant steps in denoising trajectories. It uses a hyperplanarity test in latent space, implemented via QR factorization, to allocate more steps to high‑curvature regions, and introduces the trajectory projection score α_traj as a model‑free diagnostic for flow quality. Across CIFAR‑10, LSUN Church, and Stable Diffusion v1.5, GeoSPRINT consistently outperforms uniform DDIM schedules at matched NFE budgets, improving FID scores by up to 1.93 points.

By Arpita Joshi
arXiv AI
Aug 25

ChebBooster: A Training-Free Approach for Efficient Diffusion Transformer Inference via Chebyshev-Inspired Extrapolation

ChebBooster is a training‑free extrapolation framework that accelerates Diffusion Transformers (DiTs) by using Chebyshev polynomial theory. It employs a Barycentric formulation for numerically stable evaluation and separates the process into an offline weight precomputation phase and a lightweight online application stage. Experiments on DiT‑XL/2, PixArt‑Σ, and FLUX.1‑dev show consistent visual quality gains and up to 3.68× latency speedup and 5.12× FLOPs reduction compared to existing training‑free baselines.

By Chengjie Lu, Tianchi Deng, Zhengqi He, Chengwen Luo, Xueliang Li