The paper introduces GRAS, a method that improves training‑free reward alignment for discrete diffusion models by reducing variance in guided proposals and adapting the resampling temperature during search. It achieves this without adding denoiser cost, using Rao‑Blackwellized estimates for differentiable rewards and a leave‑one‑out baseline for non‑differentiable ones. Experiments on regulatory DNA and protein design show GRAS outperforms existing training‑free techniques and rivals reward‑fine‑tuned models.
By Kwanyoung Kim
COFFEE is a plug‑and‑play framework that enables future‑aware guidance for discrete diffusion models by separating sequence dependence from the objective. It uses a target‑free carrier to absorb marginal token distributions and a compiled finite‑state model to capture how token combinations affect sequence‑level preferences, allowing global preferences to be transferred to unresolved positions without retraining the diffusion model. The framework supports both hard constraints and learned soft objectives and demonstrates strong control results across symbolic, language, and biological benchmarks.
By Hua (Edward), Xu, Dongxin Li, Gwen Yidou-Weng, Guy Van den Broeck, Wei Wang, Anji Liu
arXiv:2609.31882v2 Announce Type: replace-cross
Abstract: Reward-based diffusion fine-tuning faces practical challenges when desirable outcomes are rare or conditioning corrections are costly to esti...
By Zhengyi Guo, Jiayuan Sheng, Wenpin Tang, David D. Yao
arXiv:2502. 08006v3 Announce Type: replace-cross Abstract: Training-free guided generation is a widely used and powerful technique that allows the end user to exert further control over the generative process of flow/diffusion models.
By Zander W. Blasingame, Chen Liu
VGAS: Variance-Reduced Guidance and Adaptive Selection for Training-Free Reward Alignment in Discrete Diffusion proposes a new inference-time framework that improves steering of frozen masked discrete diffusion models. By reducing the variance of guidance estimates, applying reward tilting to clean-token logits, and adapting the selection temperature at each step, VGAS addresses three default choices in existing pipelines. Experiments on regulatory DNA, protein, and small-molecule benchmarks show that VGAS achieves the best training-free reward performance and matches or surpasses reward-fine-tuned generators.
By Kwanyoung Kim
arXiv:2608. 03929v1 Announce Type: new Abstract: Aligning diffusion models with human preferences usually relies on a sparse terminal reward evaluated on the final generated samples, presenting a severe temporal credit-assignment challenge across the multi-step denoising process.
By Yuanshen Guan, Zipeng Feng, Zhiwei Xiong, Peiqin Sun
arXiv:2606. 30376v1 Announce Type: new Abstract: Aligning generative flow models on continuous spaces via online reinforcement learning is constrained by intractable trajectory likelihoods.
By Zheming Fu, Ruizhe He, Wei Shang, Xiaoxiao Ma, Lei Wang, Chang Liu, Siming Fu
arXiv:2602. 07339v2 Announce Type: replace Abstract: Diffusion-based trajectory planners can model multi-modal driving behavior, but their iterative denoising process introduces a latency bottleneck for real-time closed-loop deployment.
By Ruturaj Reddy, Hrishav Bakul Barua, Junn Yong Loo, Thanh Thi Nguyen, Ganesh Krishnasamy
arXiv:2602. 21565v3 Announce Type: replace Abstract: Generative Flow Networks (GFlowNets) learn to sample diverse candidates in proportion to a reward function, making them well-suited for scientific discovery, where exploring multiple promising solutions is crucial.
By Seokwon Yoon, Youngbin Choi, Seunghyuk Cho, Seungbeom Lee, MoonJeong Park, Dongwoo Kim
arXiv:2606. 15359v1 Announce Type: new Abstract: Diffusion models have emerged as powerful tools for planning and control by learning multimodal distributions over actions and trajectories.
By Paolo Giaretta, Zeyang Li, Navid Azizan
arXiv:2607. 02834v1 Announce Type: new Abstract: Molecular optimization often starts from a pretrained generative model that captures a broad prior over valid molecular structures.
By Trevor Chen, Ariel Dai, Jason Yang, Riccardo De Santi, Daniel Khalil, Wenda Chu, Nate Gruver, Pranav Murugan, Alexander F. G. Goldberg, Maruan Al-Shedivat, Yisong Yue
Diffusion models have strong generative capabilities. However, their maximum likelihood training objective only focuses on reconstructing the data distribution, making it difficult to align with specific preferences.