On the spectral properties of generative denoiser Jacobians
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
The Flow has not summarised this story yet — read it at arXiv Machine Learning.
The paper investigates training objectives for denoising-based generative models, focusing on loss weighting and output parameterization such as noise-, clean image-, and velocity-based formulations. It conducts a systematic numerical study across synthetic datasets with controlled geometry and real image data, evaluating denoising accuracy via PSNR and generative quality via FID. The goal is to disentangle how training choices interact with data manifold dimensionality, model architecture, and dataset size, offering practical design insights rather than proposing a new method.
The paper investigates how current image denoisers handle textures, finding that generic denoising models often over‑smooth fine patterns and that improving texture performance requires fine‑tuning on texture‑specific datasets. However, such fine‑tuning causes the models to lose their ability to denoise the original data, indicating a trade‑off. The authors argue that dedicated, texture‑aware denoisers are necessary for accurate texture reconstruction and for downstream tasks like generative modeling.
The paper proposes a deep learning framework that learns priors for inverse problems by exploiting the relationship between proximal operators and Hamilton–Jacobi partial differential equations. Unlike existing methods that require inverting the prior after training, this approach learns the prior directly, enabling efficient evaluation in a single forward pass. Numerical experiments demonstrate the method’s effectiveness in dimensions up to 64.
The paper investigates diffusion models trained in a lazy high‑dimensional regime, extending benign overfitting theory to generative settings. By analyzing denoising score matching in a vector‑valued RKHS with an inner‑product kernel, the authors derive exact risk trajectories under gradient flow when the number of samples scales proportionally with dimensionality. These trajectories reveal three distinct phases—spectral generalization, noise‑dominated interpolation, and empirical Bayes memorization—whose interplay shapes the distribution of generated samples.
Latent Diffusion-GAN (LD‑GAN) integrates adversarial learning into diffusion models by using the pretrained VAE as an energy‑based discriminator, avoiding extra parameters and preserving the latent priors. It introduces a structural consistency energy to align encoder and decoder features, improving perceptual quality. Experiments show LD‑GAN enhances sample fidelity, sharpness, and diversity across generation tasks while maintaining efficient training dynamics.
The paper investigates how variational autoencoders (VAEs) can be made more learnable in latent diffusion by analyzing diffusability through spectral properties. It introduces the Spectrum Matching Hypothesis, which posits that latents with better diffusability should have a flattened power‑law power spectral density (Encoding Spectrum Matching) and preserve frequency‑to‑frequency semantic correspondence via the decoder (Decoding Spectrum Matching). The authors implement these ideas by matching PSDs between images and latents and applying shared spectral masking, demonstrating improved diffusion generation on CelebA and ImageNet and extending the approach to representation alignment with a DoG‑based method.