arXiv Machine Learning

Improved Distributional Diffusion Models

arXiv Machine Learning
Oct 15

Are We Overthinking One-Step Generation?

The paper introduces Priming Models, a two‑stage framework for training one‑step generative models. First, the model is trained on a reconstruction objective such as pixel‑space denoising at a fixed noise level, then it undergoes feature‑space distributional post‑training. This simple reconstruction stage alone is shown to eliminate the difficulty of distributional training, leading to significant improvements in training compute versus generation quality across tasks like class‑conditional image generation, text‑to‑image generation, and world modeling, with a state‑of‑the‑art FID of 1.19 on ImageNet 256×256 using about one‑tenth the compute of previous methods.

By Woobin Park, Jaeyeon Kim, Yilun Du, Sitan Chen, Bohyung Han
arXiv AI
Oct 19

Rethinking the design space of Vanilla Pixel Diffusion Transformers

The paper revisits pixel-space diffusion models, showing that a vanilla diffusion transformer can generate directly in pixel space by swapping velocity prediction for clean image prediction. It introduces MFlow, an adaptive parameterization that interpolates between these two prediction modes based on data representation and model capacity, and demonstrates competitive performance on ImageNet, UCF‑101, and text‑to‑image tasks. Additionally, the authors add ConvRefiner, a lightweight convolutional module trained post‑hoc to restore high‑frequency details while preserving FID.

By Soumik Mukhopadhyay, Long Mai, Abhinav Shrivastava, Aniruddha Mahapatra
arXiv AI
Oct 15

Correction Training for Diffusion Models

arXiv:2610.14136v2 Announce Type: new Abstract: Iterative sampling in diffusion models can lead to error accumulation due to exposure bias. The denoiser is trained on real images interpolated with no...

By Donghoon Ahn, Runqian Wang, Alexei A. Efros, Taesung Park, Yash Belhe
arXiv AI
6d ago

Just Image Transformers with Late-Stage Pixel Refinement

The paper revisits JiT, a latent‑free diffusion transformer that generates images by predicting the final image. By adding simple convolutional pixel‑pathway layers after training, the authors significantly improve generation quality, showing that these layers can be activated only in the last 20% of the denoising process. On ImageNet@256, the method achieves an FID of 1.78, better than JiT’s 2.34, without extra losses.

By Denis Lukovnikov, Andreas M\"uller, Asja Fischer
arXiv AI
Nov 6

Volterra Generative Models via Markovian Lifting

The paper introduces Volterra generative models that extend diffusion-based generative models by incorporating Volterra processes, which provide more flexible temporal dependence than traditional Brownian-motion-driven Markov processes. To address the non-Markovian challenges, the authors develop Markovian approximations and Gaussian conditioning, reducing the denoising problem to the original data dimension and enabling connections to variance-exploding and variance-preserving frameworks. The resulting models, trained from scratch or with pretrained denoisers, achieve competitive FID scores on MNIST, conditional CIFAR-10, and ImageNet datasets, demonstrating the practical viability of the approach.

By Bingyan Han
arXiv AI
Nov 5

PEEK: One-Step-Look-Ahead Exposure Bias Correction for Diffusion Sampling

The paper introduces PEEK, a training‑free deterministic sampler plugin that corrects exposure bias in diffusion image generators by performing a provisional one‑step update and interpolating between predictions at a look‑ahead state. PEEK evaluates the denoiser at this higher‑signal noisy state, thereby producing more image‑like predictions and reducing the train‑test gap that often leads to blurry or poorly detailed outputs. Experiments on eight pixel‑space and latent‑space diffusion models—including unconditional, label‑guided, and text‑guided settings—show consistent improvements over NFE‑matched baselines, with FID gains up to 49.3 % and minimal additional computation.

By Yunwei Bai, Ying Kiat Tan, Tsuhan Chen
arXiv Computer Vision
Sep 28

PhoenixSR: Generative Heterogeneous Distillation Unleashes Efficient Models for Real-World Super-Resolution

arXiv:2609.30988v1 Announce Type: new Abstract: Real-world image super-resolution (SR) requires recovering perceptually realistic high-resolution images from complex low-resolution observations while...

By Xin Di, Mingyu Shi, Yuanfei Bao, Long Peng, Yue Zhao, Jiaming Guo, Renjing Pei, Xueyang Fu, Yang Cao, Zheng-Jun Zha