Diffusion models are increasingly used as controllable samplers, whose generations can be steered at inference time according to a chosen reward function. While such rewards are typically defined on individual samples, for many applications it is desirable to steer according to distribution-level rewards, for example to calibrate with population-level information or to encourage diversity.
arXiv:2608. 08770v1 Announce Type: cross Abstract: Diffusion models are increasingly used as controllable samplers, whose generations can be steered at inference time according to a chosen reward function.
arXiv:2609.37227v1 Announce Type: new Abstract: Inference-time steering adapts pretrained diffusion and flow-based models to new tasks, e.g., to generate samples from a conditional distribution or sa...
arXiv:2606. 02884v1 Announce Type: cross Abstract: Reward guidance algorithms steer a learned generative process toward the reward-tilted measure at inference time.
arXiv:2606. 13240v1 Announce Type: cross Abstract: A key strength of diffusion models lies in their flexibility, since their outputs can be controlled at sampling time through guidance.
The paper introduces GRAS, a method that improves training‑free reward alignment for discrete diffusion models by reducing variance in guided proposals and adapting the resampling temperature during search. It achieves this without adding denoiser cost, using Rao‑Blackwellized estimates for differentiable rewards and a leave‑one‑out baseline for non‑differentiable ones. Experiments on regulatory DNA and protein design show GRAS outperforms existing training‑free techniques and rivals reward‑fine‑tuned models.
Fenchel Tilt Flow Control (FTFC) is a new method for fine‑tuning pretrained generative models to arbitrary preference functions. It decouples utility optimization from model fitting by first learning reward and density‑ratio weights on pretrained samples, then freezing these weights to adjust a diffusion or flow model in a single importance‑weighted stage. The approach supports general f‑divergence penalties, achieves exact duality for concave utilities, and demonstrates up to 20× efficiency gains while outperforming baselines on image and molecule generation tasks.
arXiv:2607. 07693v1 Announce Type: cross Abstract: Reinforcement learning from human feedback (RLHF) has emerged as a powerful paradigm for aligning generative models with human preferences.
arXiv:2602. 03211v2 Announce Type: replace-cross Abstract: Diffusion models have demonstrated strong generative performance; however, generated samples often fail to fully align with human intent.
VGAS: Variance-Reduced Guidance and Adaptive Selection for Training-Free Reward Alignment in Discrete Diffusion proposes a new inference-time framework that improves steering of frozen masked discrete diffusion models. By reducing the variance of guidance estimates, applying reward tilting to clean-token logits, and adapting the selection temperature at each step, VGAS addresses three default choices in existing pipelines. Experiments on regulatory DNA, protein, and small-molecule benchmarks show that VGAS achieves the best training-free reward performance and matches or surpasses reward-fine-tuned generators.
arXiv:2507. 08390v5 Announce Type: replace Abstract: Discrete diffusion models have recently emerged as strong alternatives to autoregressive language models, matching their performance through large-scale training.
The paper investigates training diffusion models to sample from distributions defined by unnormalized densities or energy functions. It benchmarks various diffusion-structured inference techniques, including simulation-based variational methods and off-policy approaches such as continuous generative flow networks, highlighting their relative strengths and challenging some prior claims. Additionally, the authors introduce a new exploration strategy for off-policy methods that employs local search in the target space with a replay buffer, demonstrating improved sample quality across multiple target distributions.
The paper introduces a method for specificity‑aware diffusion steering that suppresses undesired samples while preserving desired ones. By formulating the problem as a target‑design task, it derives a time‑dependent target distribution based on overlap between positive and negative reference distributions, and samples from it using a variance‑reduced Sequential Monte Carlo (SMC) sampler. Experiments on synthetic, class‑contrastive, text‑to‑image, and peptide‑MHC tasks demonstrate reduced mode shift, improved sampling stability, and better suppression of undesired regions compared to negative‑guidance baselines.