arXiv:2608. 20011v1 Announce Type: new Abstract: Preference optimization is a standard alignment method for generative models, yet extending it to continuous-time dynamics remains non-trivial.
arXiv:2609.09905v1 Announce Type: cross Abstract: Preference alignment for flow and diffusion models now spans online reinforcement learning and offline preference optimization, but the relation betw...
arXiv:2601. 22495v2 Announce Type: replace Abstract: Fine-tuning flow matching models is a central challenge in settings with limited data, evolving distributions, or computational constraints.
PMosFM introduces a preconditioned manifold matching framework that enables one‑step physics‑constrained generation by encoding constraints in a manifold decoder. The method learns transport in intrinsic coordinates, eliminating the need for residual losses or trajectory unrolling, and employs a geometric preconditioner and covariance transform to improve conditioning. Experiments demonstrate reduced training and sampling time compared to multi‑step baselines while maintaining comparable physical and distributional fidelity.
arXiv:2606. 11075v1 Announce Type: new Abstract: Aligning text-to-image flow matching models with human preferences via direct reward backpropagation is sample-efficient but hampered by two well-known pathologies: activations cannot be stored across the full sampling trajectory at modern model scale, and chained Jacobian products across steps inflate the reward gradient as it travels back to early indices.
Fenchel Tilt Flow Control (FTFC) is a new method for fine‑tuning pretrained generative models to arbitrary preference functions. It decouples utility optimization from model fitting by first learning reward and density‑ratio weights on pretrained samples, then freezing these weights to adjust a diffusion or flow model in a single importance‑weighted stage. The approach supports general f‑divergence penalties, achieves exact duality for concave utilities, and demonstrates up to 20× efficiency gains while outperforming baselines on image and molecule generation tasks.
arXiv:2604. 17415v3 Announce Type: replace-cross Abstract: Reward-based fine-tuning steers a pretrained diffusion or flow-based generative model toward higher-reward samples while remaining close to the pretrained model.
Efficient Adjoint Matching (EAM) is a new method for reward fine‑tuning of pretrained diffusion models that reduces computational cost by reformulating the stochastic optimal control problem with a linear base drift and a modified terminal cost. This change eliminates the need for extensive stochastic simulation of full generative trajectories and backward ODE simulation of the adjoint state, allowing training‑time sampling with a few‑step deterministic ODE solver and a closed‑form adjoint solution. On standard text‑to‑image reward fine‑tuning benchmarks, EAM converges up to four times faster than the previous Adjoint Matching approach while matching or surpassing it on metrics such as PickScore, ImageReward, HPSv2.1, CLIPScore, and Aesthetics.
arXiv:2609.05727v1 Announce Type: cross Abstract: We develop Newton Matching, a unified framework for fine-tuning and sampling in generative modeling. The target is $\pi\propto\mu e^{\tau r}$, where...
arXiv:2610.02260v1 Announce Type: new Abstract: Flow matching models excel at generative modeling, and many downstream applications require their samples to satisfy prescribed constraints, such as ob...
arXiv:2607. 08781v1 Announce Type: cross Abstract: The coupling in flow matching -- the rule pairing noise vectors with data points -- is typically treated as a computational choice.
The paper introduces Noise Policy Optimization (NPO), a method for aligning few-step generative models with black-box reward signals. NPO learns an amortized flow policy that transforms initial Gaussian noise into a reward-tilted distribution, allowing the pretrained generator to produce higher-reward outputs without fine-tuning. The approach achieves superior reward-quality trade-offs across image, language, and protein generation tasks while drastically reducing inference cost compared to test-time search methods.
arXiv:2606. 02521v1 Announce Type: new Abstract: One-step text-to-image generators are attractive for deployment because they generate an image with a single forward pass, but preference finetuning them remains difficult: standard alignment methods often rely on policy likelihoods, denoising trajectories, differentiable reward gradients, or test-time optimization.