← Back to all news
arXiv Machine Learning

Prediction--Loss Alignment for Sampler--Robust Flow Matching Training

The paper examines the common practice in diffusion and flow matching of predicting the clean signal, converting it to a velocity, and training with a velocity-space loss. It shows that this conversion can cause unstable optimization due to singular endpoint amplification, but that prediction–loss alignment removes this source of non‑integrability and ensures a finite second moment for all timesteps, even with uniform sampling. Experiments confirm that aligned objectives remain trainable across different samplers, reconciling theoretical concerns with empirical success.

Related stories

arXiv AI
Oct 5

AREX: Affine-Residual Exponential Integrator for Few-Step Sampling in Flow Matching

AREX is a training‑free sampler for pretrained flow matching models that leverages the target mean and covariance to analytically capture part of the sampling dynamics. By showing that the velocity field of a moment‑matched Gaussian target is the $L^2$‑optimal affine approximation to the marginal velocity field, AREX decomposes the learned dynamics into an affine component—determined by the first two target moments—and a neural residual term. The affine component is integrated explicitly with a matrix‑valued propagator, leaving only the residual term to be integrated numerically, which consistently improves sample fidelity in few‑step image and text‑to‑image generation tasks without retraining the underlying model.

By Shizheng Lin, Soon Hoe Lim, N. Benjamin Erichson
diffusion
More like this →
arXiv AI
Sep 2

Training-Free Refinement of Flow Matching with Divergence-based Sampling

The paper introduces Flow Divergence Sampler (FDS), a training‑free method that refines intermediate states in flow‑matching models by using the divergence of the marginal velocity field to detect and correct misguidance toward low‑density regions. FDS operates during inference, requires no additional training, and can be applied as a plug‑and‑play module with standard solvers and existing flow backbones. Experiments show that FDS consistently improves fidelity in tasks such as text‑to‑image synthesis and inverse problems.

By Yeonwoo Cha, Jaehoon Yoo, Semin Kim, Yunseo Park, Jinhyeon Kwon, Seunghoon Hong
diffusion
More like this →
arXiv Machine Learning
Oct 28

Cumulative Marginal Matching for One-Step Diffusion

The paper introduces Cumulative Marginal Matching (CMM), a new objective for training few‑step diffusion and flow‑matching models. CMM treats flow‑matching as a sequence of intermediate marginal distributions and directly optimizes the cumulative Wasserstein‑2 discrepancy between the base and few‑step models across the entire trajectory. It derives a tractable surrogate for this objective, shows that Shortcut‑CSL is a first‑order approximation of CMM, and reports improved generation quality on standard benchmarks under the same training budget.

By Paribesh Regmi, Rui Li
diffusionbenchmarkssafety
More like this →
arXiv AI
Oct 28

Preconditioned Flow Matching

Preconditioned Flow Matching introduces a precondition-then-match framework that transforms target distributions into more isotropic representations before training flow matching models. By reshaping intermediate distributions, the method improves conditioning, leading to better low-eigenvalue recovery and enhanced performance metrics such as FID, MMD, precision, and recall across Gaussian, Gaussian-mixture, and high-resolution image datasets. The approach is theoretically justified and empirically validated, with ablations showing gains stem from improved geometry rather than extra parameters.

By Shadab Ahamed, Eshed Gal, Md Shahriar Rahim Siddiqui, Simon Ghyselincks, Moshe Eliasof, Eldad Haber
diffusion
More like this →