arXiv:2509. 05441v4 Announce Type: replace-cross Abstract: Latent generative models compress images into learned embeddings prior to synthesis, and the generation quality critically depends on how faithfully these embeddings preserve visual detail.
arXiv:2411.05005v2 Announce Type: replace-cross Abstract: Beyond high-fidelity image synthesis, diffusion models have recently exhibited promising results in dense visual perception tasks. However, m...
Built on pretrained vision foundation models (VFMs), representation autoencoders (RAEs) have recently emerged as a promising approach for constructing semantically rich latent spaces for image generation. However, their reconstruction quality often remains suboptimal, largely because deep VFM representations do not preserve sufficient fine-grained visual detail.
arXiv:2609.24919v1 Announce Type: new Abstract: Recent advances in pixel-space diffusion models have narrowed the image quality gap with latent-space diffusion, but still converge more slowly and lag...
The paper investigates how variational autoencoders (VAEs) can be made more learnable in latent diffusion by analyzing diffusability through spectral properties. It introduces the Spectrum Matching Hypothesis, which posits that latents with better diffusability should have a flattened power‑law power spectral density (Encoding Spectrum Matching) and preserve frequency‑to‑frequency semantic correspondence via the decoder (Decoding Spectrum Matching). The authors implement these ideas by matching PSDs between images and latents and applying shared spectral masking, demonstrating improved diffusion generation on CelebA and ImageNet and extending the approach to representation alignment with a DoG‑based method.
UltraDiff is a new framework for high‑resolution image generation with Latent Diffusion Models that tackles the trade‑off between computational cost and fidelity caused by latent compression. It jointly learns latent diffusion and then distills a marginal score using a dual‑branch DiT called MixTeacher, which models both f8 and f32 distributions and resolves compressed‑space ambiguity with high‑fidelity priors. Vector‑field distillation in the native f32 space forces the student model to learn the teacher’s conditional expectation, filtering out compression artifacts and achieving f8‑level quality while retaining f32 efficiency—generating a 2K image in 6.2 seconds on a single A100 GPU, over 40× faster than the f8 baseline.
arXiv:2606. 09056v1 Announce Type: cross Abstract: Video generative models have become increasingly powerful, but long-range consistency remains challenging to achieve because even a few dozen frames require impractically long transformer sequence lengths.
arXiv:2608. 00111v1 Announce Type: cross Abstract: Image restoration quality can be evaluated along three complementary facets: pixel-level fidelity, human perception, and downstream machine preference.
The paper proposes a two-stage training strategy for Implicit Neural Representations (INRs) that addresses spectral bias by using a neighbor-aware soft mask to emphasize high-frequency details early in training. In the first stage, the mask assigns higher weights to pixels with strong local variations, encouraging the network to focus on fine edges and textures. The second stage transitions to full-image training, and experiments show consistent improvements in reconstruction quality across existing INR methods.
V-Co investigates visual co-denoising for pixel-space diffusion models, using a unified JiT-based framework to isolate key design choices. The study identifies two essential components: a dual-stream architecture with flexible cross-stream interaction and a perceptual-drifting hybrid loss combined with RMS-based feature rescaling for stronger semantic supervision. Experiments on ImageNet-256 demonstrate that V-Co surpasses baseline pixel-space diffusion and strong prior pixel-diffusion methods at comparable model sizes while requiring fewer training epochs.
The paper investigates how fine‑tuning pretrained visual encoders for faithful image reconstruction affects diffusion models that operate in the resulting latent space. It finds that such fine‑tuning reduces the effective dimensionality of the latent representation, causing standard velocity‑prediction flow‑matching to fit noise outside the low‑dimensional signal manifold and making optimization inefficient. Consequently, the authors propose using a clean‑data ($oldsymbol{x}_{0}$) parameterization, which focuses learning on the signal manifold and consistently improves text‑to‑image generation across multiple strong‑reconstruction encoders.
The paper introduces MatLat, a material latent space designed for generating high‑quality physically‑based rendering (PBR) textures on 3D meshes. It fine‑tunes a pretrained VAE to incorporate new material channels while preserving latent distribution, and adds a locality‑preserving regularization that aligns cropped latent patches with image regions. Experiments and ablation studies show that these components together improve PBR texture fidelity and outperform prior baselines.
UGDiff introduces an uncertainty-guided diffusion paradigm for single-image super-resolution, aiming to improve the perception‑distortion trade‑off. The method estimates reconstruction uncertainty of latent features from a high‑fidelity image and uses this uncertainty, along with diffusion sampler posterior variance, to selectively restore high‑frequency details in uncertain regions while preserving fidelity elsewhere. Experiments show that UGDiff outperforms state‑of‑the‑art diffusion‑based SR methods.