arXiv Machine Learning
Jun 19
Meta Flow Maps enable scalable reward alignment
arXiv:2601. 14430v2 Announce Type: replace-cross Abstract: Controlling generative models is computationally expensive.
How to Guide Your Flow: Few-Step Alignment via Flow Map Reward Guidance
arXiv:2604. 27147v3 Announce Type: replace-cross Abstract: In generative modeling, we often wish to produce samples that maximize a user-specified reward such as aesthetic quality or alignment with human preferences, a problem known as \textit{guidance}.
Related stories
arXiv Machine Learning
Jun 10
Exploring the Design Space of Reward Backpropagation for Flow Matching
arXiv:2606. 11075v1 Announce Type: new Abstract: Aligning text-to-image flow matching models with human preferences via direct reward backpropagation is sample-efficient but hampered by two well-known pathologies: activations cannot be stored across the full sampling trajectory at modern model scale, and chained Jacobian products across steps inflate the reward gradient as it travels back to early indices.
arXiv Machine Learning
Sep 24
WTF?! Simulation-Free Reinforcement Learning with Wasserstein-Tilted Flow Maps
The paper introduces Wasserstein‑Tilted Flow Maps (WTF), a simulation‑free reinforcement learning method that fine‑tunes pre‑trained flow‑based generative models by adding an optimal transport regularizer derived from the model’s drift. Unlike traditional KL‑reward tilting, WTF transports individual samples toward higher reward, framing the problem as a deterministic optimal control task on the flow map. Experiments on ImageNet‑256 and text‑to‑image demonstrate that WTF achieves higher reward and comparable or better diversity while reducing training compute by up to 280×.
arXiv AI
Jun 2
Reward Score Matching: Unifying Reward-based Fine-tuning for Flow and Diffusion Models
arXiv:2604. 17415v3 Announce Type: replace-cross Abstract: Reward-based fine-tuning steers a pretrained diffusion or flow-based generative model toward higher-reward samples while remaining close to the pretrained model.
arXiv AI
Jul 2
Flow-Map GRPO: Reinforcement Learning for Few-Step Flow-Map Generators via Anchored Stochastic Composition
arXiv:2607. 00535v1 Announce Type: cross Abstract: Few-step flow-map generators, such as consistency models and MeanFlow, accelerate sampling by directly learning long-range transport maps between noise and data.
arXiv AI
Sep 1
CF-VLA: Efficient Coarse-to-Fine Action Generation for Vision-Language-Action Policies
CF‑VLA introduces a two‑stage coarse‑to‑fine approach for vision‑language‑action policies, replacing multi‑step sampling with a coarse initialization that constructs an action‑aware starting point and a single‑step refinement that corrects residual errors. The coarse stage learns a conditional posterior over endpoint velocity to transform Gaussian noise into a structured initialization, while the fine stage performs a fixed‑time refinement. Experiments on CALVIN and LIBERO demonstrate that CF‑VLA achieves a strong efficiency‑performance trade‑off, reducing action sampling latency by 75.4 % and achieving an 83.0 % real‑robot success rate, outperforming existing NFE=2 methods and matching or surpassing NFE=10 baselines.
arXiv AI
Jul 1
Finite Difference Flow Optimization for RL Post-Training of Text-to-Image Models
arXiv:2603. 12893v2 Announce Type: replace-cross Abstract: Reinforcement learning (RL) has become a standard technique for post-training diffusion-based image synthesis models, as it enables learning from reward signals to explicitly improve desirable aspects such as image quality and prompt alignment.
arXiv Machine Learning
Jun 30
FlowAWR: Online Adaptive Flow Reinforcement via Advantage-Weighted Rectification
arXiv:2606. 30376v1 Announce Type: new Abstract: Aligning generative flow models on continuous spaces via online reinforcement learning is constrained by intractable trajectory likelihoods.
arXiv AI
Jun 3
Greed is Good: A Unifying Perspective on Guided Generation
arXiv:2502. 08006v3 Announce Type: replace-cross Abstract: Training-free guided generation is a widely used and powerful technique that allows the end user to exert further control over the generative process of flow/diffusion models.
arXiv Machine Learning
Jul 31
S-GRPO: Unified Post-Training for Large Vision-Language Models
arXiv:2604. 16557v2 Announce Type: replace Abstract: Current post-training methodologies for adapting Large Vision-Language Models (LVLMs) generally fall into two paradigms: Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL).
arXiv Machine Learning
Jun 2
Drifting Preference Optimization for One-Step Generative Models
arXiv:2606. 02521v1 Announce Type: new Abstract: One-step text-to-image generators are attractive for deployment because they generate an image with a single forward pass, but preference finetuning them remains difficult: standard alignment methods often rely on policy likelihoods, denoising trajectories, differentiable reward gradients, or test-time optimization.
arXiv AI
Aug 11
RL-Native Distillation: Exploiting Scored Trajectories for Few-Step Image Generation
arXiv:2608. 09226v1 Announce Type: cross Abstract: Efficient text-to-image generation requires both reinforcement-learning (RL)-based reward alignment and few-step distillation, yet these procedures are typically performed sequentially, increasing training cost and risking the loss of reward gains during compression.