arXiv:2411.05005v2 Announce Type: replace-cross
Abstract: Beyond high-fidelity image synthesis, diffusion models have recently exhibited promising results in dense visual perception tasks. However, m...
By Shuhong Zheng, Zhipeng Bao, Ruoyu Zhao, Martial Hebert, Yu-Xiong Wang
Built on pretrained vision foundation models (VFMs), representation autoencoders (RAEs) have recently emerged as a promising approach for constructing semantically rich latent spaces for image generation. However, their reconstruction quality often remains suboptimal, largely because deep VFM representations do not preserve sufficient fine-grained visual detail.
arXiv:2609.24919v1 Announce Type: new
Abstract: Recent advances in pixel-space diffusion models have narrowed the image quality gap with latent-space diffusion, but still converge more slowly and lag...
By Yongsheng Yu, Wei Xiong, Yichen Sheng, Shiqiu Liu, Jiebo Luo
The paper investigates how variational autoencoders (VAEs) can be made more learnable in latent diffusion by analyzing diffusability through spectral properties. It introduces the Spectrum Matching Hypothesis, which posits that latents with better diffusability should have a flattened power‑law power spectral density (Encoding Spectrum Matching) and preserve frequency‑to‑frequency semantic correspondence via the decoder (Decoding Spectrum Matching). The authors implement these ideas by matching PSDs between images and latents and applying shared spectral masking, demonstrating improved diffusion generation on CelebA and ImageNet and extending the approach to representation alignment with a DoG‑based method.
By Mang Ning, Mingxiao Li, Le Zhang, Lanmiao Liu, Matthew B. Blaschko, Albert Ali Salah, Itir Onal Ertugrul
UltraDiff is a new framework for high‑resolution image generation with Latent Diffusion Models that tackles the trade‑off between computational cost and fidelity caused by latent compression. It jointly learns latent diffusion and then distills a marginal score using a dual‑branch DiT called MixTeacher, which models both f8 and f32 distributions and resolves compressed‑space ambiguity with high‑fidelity priors. Vector‑field distillation in the native f32 space forces the student model to learn the teacher’s conditional expectation, filtering out compression artifacts and achieving f8‑level quality while retaining f32 efficiency—generating a 2K image in 6.2 seconds on a single A100 GPU, over 40× faster than the f8 baseline.
By Jingjing Ren, Haitian Zheng, Haoyu Chen, Connelly Barnes, Zhifei Zhang, Yuqian Zhou, Zhe Lin, Lei Zhu
arXiv:2606. 09056v1 Announce Type: cross Abstract: Video generative models have become increasingly powerful, but long-range consistency remains challenging to achieve because even a few dozen frames require impractically long transformer sequence lengths.
By Ishaan Preetam Chandratreya, David Charatan, Basile Van Hoorick, Sergey Zakharov, Vitor Guizilini, Phillip Isola, Vincent Sitzmann