arXiv Machine Learning

Adversarial Learning of Classifier-Free Guidance Schedules

arXiv:2608. 14038v1 Announce Type: new Abstract: Modern text-to-image diffusion models rely on classifier-free guidance (CFG) to achieve high image fidelity and text alignment.

arXiv Machine Learning
1d ago

Learned End-to-End Guidance Schedules for Diffusion Models

The paper introduces Learned End-to-End Guidance Schedules (LEEGS) for diffusion models, which train a time‑dependent guidance schedule to balance data quality and requirement satisfaction while reducing sampling steps. LEEGS minimizes the guidance function over a small set of examples using stochastic gradient descent and employs a gradient approximation to cut training time by a factor of four. Experiments on tasks such as image inpainting, noisy image inverse problems, face‑ID‑guided generation, and PDE problems show that LEEGS outperforms baselines at the same computational budget or matches constant guidance with only 10% of the steps.

By Aneesh Barthakur, Mathias Niepert, Luiz F. O. Chamon
arXiv AI
Sep 16

Efficient Text-to-Image Generation: An Adaptive Step Schedule Controller for Diffusion Models

The paper introduces an adaptive step schedule controller for text‑to‑image diffusion models, allowing the number of denoising steps to vary based on the complexity of the input prompt. By mixing step schedules of different sizes and monitoring error discrepancies at each timestep, the method switches schedules to maintain image quality while reducing inference time. Experiments on COCO and DiffusionDB demonstrate that this approach achieves faster generation without sacrificing visual fidelity.

By Kuluhan Binici, Cihan Acar, Shivam Aggarwal, Siying Liu, Tulika Mitra
arXiv Machine Learning
Aug 27

Continuous Adversarial Flow Models

The paper introduces continuous adversarial flow models, a continuous-time flow framework trained with an adversarial objective that replaces the fixed mean-squared-error criterion of flow matching. By incorporating a learned discriminator, the method guides training toward a different generalized distribution, yielding samples more closely aligned with the target data distribution. Applied as a post‑training step, it markedly improves ImageNet 256px generation metrics—reducing the guidance‑free FID of latent‑space SiT from 8.26 to 3.63 and of pixel‑space JiT from 7.17 to 3.57—and also enhances guided generation and text‑to‑image benchmarks.

By Shanchuan Lin, Ceyuan Yang, Zhijie Lin, Hao Chen, Haoqi Fan
arXiv AI
3d ago

Image AID via continuous-time reinforcement learning

The paper introduces Amortized Inpainting with Diffusion (AID), a method that keeps a pretrained diffusion backbone fixed and trains a small reusable guidance module offline for image inpainting. AID formulates the problem as deterministic guidance with a supervised terminal objective, derives an auxiliary Gaussian formulation to make it learnable, and proves that solving the randomized problem recovers the optimal deterministic guidance field. Experiments on AFHQv2, FFHQ, and ImageNet show that AID improves the quality–speed trade‑off over strong baselines while adding less than one percent trainable overhead.

By Yilie Huang, Xun Yu Zhou
arXiv Computer Vision
Sep 7

PAPT++: Risk-Aware Adversarial Tuning and Generation for Single Domain Generalization

PAPT++ is a risk‑aware adversarial generation‑training framework designed to improve single domain generalization. It learns diverse semantic reference images per class and uses them as denoising targets in classifier‑guided diffusion synthesis, thereby generating challenging yet semantically consistent samples. These samples are iteratively combined with source data to update the classifier, progressively exposing it to difficult variations and enhancing generalization performance on standard benchmarks.

By Zhipeng Xu, De Cheng, Xinyang Jiang, Lingfeng He, Huaijie Wang, Dongsheng Li, Nannan Wang, Xinbo Gao