Improved Distributional Diffusion Models
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
The Flow has not summarised this story yet — read it at arXiv Machine Learning.
The paper introduces Priming Models, a two‑stage framework for training one‑step generative models. First, the model is trained on a reconstruction objective such as pixel‑space denoising at a fixed noise level, then it undergoes feature‑space distributional post‑training. This simple reconstruction stage alone is shown to eliminate the difficulty of distributional training, leading to significant improvements in training compute versus generation quality across tasks like class‑conditional image generation, text‑to‑image generation, and world modeling, with a state‑of‑the‑art FID of 1.19 on ImageNet 256×256 using about one‑tenth the compute of previous methods.
The paper revisits pixel-space diffusion models, showing that a vanilla diffusion transformer can generate directly in pixel space by swapping velocity prediction for clean image prediction. It introduces MFlow, an adaptive parameterization that interpolates between these two prediction modes based on data representation and model capacity, and demonstrates competitive performance on ImageNet, UCF‑101, and text‑to‑image tasks. Additionally, the authors add ConvRefiner, a lightweight convolutional module trained post‑hoc to restore high‑frequency details while preserving FID.
arXiv:2610.14136v2 Announce Type: new Abstract: Iterative sampling in diffusion models can lead to error accumulation due to exposure bias. The denoiser is trained on real images interpolated with no...
arXiv:2610.19305v1 Announce Type: new Abstract: One-step diffusion distillation achieves strong performance on small datasets but remains insufficient for high-resolution or complex text-to-image set...
The paper revisits JiT, a latent‑free diffusion transformer that generates images by predicting the final image. By adding simple convolutional pixel‑pathway layers after training, the authors significantly improve generation quality, showing that these layers can be activated only in the last 20% of the denoising process. On ImageNet@256, the method achieves an FID of 1.78, better than JiT’s 2.34, without extra losses.
arXiv:2607. 19332v1 Announce Type: new Abstract: Generative models have undergone many generations of evolution, from VAEs/GANs to diffusion/flow matching.