VGAS: Variance-Reduced Guidance and Adaptive Selection for Training-Free Reward Alignment in Discrete Diffusion proposes a new inference-time framework that improves steering of frozen masked discrete diffusion models. By reducing the variance of guidance estimates, applying reward tilting to clean-token logits, and adapting the selection temperature at each step, VGAS addresses three default choices in existing pipelines. Experiments on regulatory DNA, protein, and small-molecule benchmarks show that VGAS achieves the best training-free reward performance and matches or surpasses reward-fine-tuned generators.
By Kwanyoung Kim
arXiv:2608. 03929v1 Announce Type: new Abstract: Aligning diffusion models with human preferences usually relies on a sparse terminal reward evaluated on the final generated samples, presenting a severe temporal credit-assignment challenge across the multi-step denoising process.
By Yuanshen Guan, Zipeng Feng, Zhiwei Xiong, Peiqin Sun
D-DOIT (Discrete Doob-Oriented Inference-time Transformation) is a training‑free method that adapts discrete diffusion models to generic rewards by applying Doob's h‑transform to the reverse diffusion kernel. It reweights reverse transition probabilities using only reward values, avoiding costly future rollouts, and optionally refines trajectories near the end with a best‑of‑K strategy. Experiments on regulatory DNA design and protein inverse folding show that D-DOIT outperforms other training‑free guidance baselines, improving enhancer activity, cell‑type specificity, and protein folding success rates while maintaining sequence naturalness.
By Jieke Wu, Qijie Zhu, Weimin Wu, Zeqi Ye, Minshuo Chen, Han Liu
The paper introduces Noise Policy Optimization (NPO), a method for aligning few-step generative models with black-box reward signals. NPO learns an amortized flow policy that transforms initial Gaussian noise into a reward-tilted distribution, allowing the pretrained generator to produce higher-reward outputs without fine-tuning. The approach achieves superior reward-quality trade-offs across image, language, and protein generation tasks while drastically reducing inference cost compared to test-time search methods.
By Woocheol Shin, Taeyoung Yun, Jiwon Sung, Hyeongyu Kang, Jeongmin Park, Jaewoo Lee, Kyuil Sim, Dohyun Kim, Yutong He, Jinkyoo Park
COFFEE is a plug‑and‑play framework that enables future‑aware guidance for discrete diffusion models by separating sequence dependence from the objective. It uses a target‑free carrier to absorb marginal token distributions and a compiled finite‑state model to capture how token combinations affect sequence‑level preferences, allowing global preferences to be transferred to unresolved positions without retraining the diffusion model. The framework supports both hard constraints and learned soft objectives and demonstrates strong control results across symbolic, language, and biological benchmarks.
By Hua (Edward), Xu, Dongxin Li, Gwen Yidou-Weng, Guy Van den Broeck, Wei Wang, Anji Liu
arXiv:2603. 14504v2 Announce Type: replace-cross Abstract: Optimizing the noise samples of diffusion and flow models is an increasingly popular approach to align these models to target rewards at inference time.
By Niklas Schweiger, Daniel Cremers, Karnik Ram