arXiv Machine Learning

On the Diffusibility of High-Dimensional Latents

The paper investigates how fine‑tuning pretrained visual encoders for faithful image reconstruction affects diffusion models that operate in the resulting latent space. It finds that such fine‑tuning reduces the effective dimensionality of the latent representation, causing standard velocity‑prediction flow‑matching to fit noise outside the low‑dimensional signal manifold and making optimization inefficient. Consequently, the authors propose using a clean‑data ($oldsymbol{x}_{0}$) parameterization, which focuses learning on the signal manifold and consistently improves text‑to‑image generation across multiple strong‑reconstruction encoders.

arXiv AI
Oct 19

Rethinking the design space of Vanilla Pixel Diffusion Transformers

The paper revisits pixel-space diffusion models, showing that a vanilla diffusion transformer can generate directly in pixel space by swapping velocity prediction for clean image prediction. It introduces MFlow, an adaptive parameterization that interpolates between these two prediction modes based on data representation and model capacity, and demonstrates competitive performance on ImageNet, UCF‑101, and text‑to‑image tasks. Additionally, the authors add ConvRefiner, a lightweight convolutional module trained post‑hoc to restore high‑frequency details while preserving FID.

By Soumik Mukhopadhyay, Long Mai, Abhinav Shrivastava, Aniruddha Mahapatra
arXiv Computer Vision
Sep 28

FuseReg: Regularizing Layer Fusion Mitigates the Reconstruction-Generation Gap in Representation Autoencoders

arXiv:2609.31620v1 Announce Type: new Abstract: Representation autoencoders (RAEs) reuse features from a pretrained visual encoder as reconstruction and diffusion latents, integrating strong visual r...

By Hongyang Du, Yunfei Xie, Junjie Ye, Jiawei Yang, Xiaoyan Cong, Haodong Zhang, Yongchao Huang, Haiyu Wu, Zongxia Li, Shihang Gui, Dawei Liu, Runhao Li, Jingcheng Ni, Chen Wei, Randall Balestriero, Yue Wang
arXiv Computer Vision
Sep 18

Training Flow Matching: The Role of Weighting and Parameterization

The paper investigates training objectives for denoising-based generative models, focusing on loss weighting and output parameterization such as noise-, clean image-, and velocity-based formulations. It conducts a systematic numerical study across synthetic datasets with controlled geometry and real image data, evaluating denoising accuracy via PSNR and generative quality via FID. The goal is to disentangle how training choices interact with data manifold dimensionality, model architecture, and dataset size, offering practical design insights rather than proposing a new method.

By Anne Gagneux, S\'egol\`ene Martin, R\'emi Gribonval, Mathurin Massias
arXiv Computer Vision
Oct 12

InstantFusion: A Shared Latent Interface for Heterogeneous Image Generators

InstantFusion introduces a lightweight shared latent interface that enables cross-model state translation between heterogeneous image generators. By training model-specific Latent AutoEncoders to project intermediate denoising states into a common latent space, the method aligns states from the same image at matched noise levels and synchronizes their local denoising velocities. This allows seamless denoising transitions across frozen generators, supporting applications such as acceleration, multi-preference composition, on-policy distillation, and cross-model composition.

By Xuanlang Dai, Jiazi Bu, Yujie Zhou, Yuhong Liu, Beichen Zhang, Penghui Yang, Ziyu Liu, Yuhang Zang
Hugging Face Trending Papers
Aug 6

Energy-Guided Flow Matching

Pixel-space generative models bypass lossy latent compression, yet necessitate joint learning of global structure and fine-grained details in a high-dimensional space. Standard flow matching interpolates noise toward a fixed clean-image endpoint, leaving the spectral evolution to be learned implicitly.

arXiv AI
Oct 28

Beyond the Estimate: Wiener-Informed Decoding for Easier Training of Pixel-Space Flow Models

The paper introduces Wiener-informed Decoding (WiD), a method that leverages frozen Wiener estimates and statistical knowledge to improve the training of pixel-space flow models. By integrating a trainable Wiener prediction path with coefficient interfaces that inject features into the native decoder, WiD accelerates training, enhances generation quality, and yields smoother gradient trajectories across various model scales and decoder architectures. Experiments on ImageNet 256×256 show that PixelDiT-XL with WiD achieves an FID of 1.53 after 500 epochs, matching the quality of the baseline while requiring 37.5% fewer training epochs.

By Ziqiao Wang, Yang You