arXiv Machine Learning
Jun 4
Geometry-Preserving Encoder/Decoder in Latent Generative Models
arXiv:2501. 09876v3 Announce Type: replace-cross Abstract: Generative modeling aims to generate new data samples that resemble a given dataset.
Generalization in VAE and Diffusion Models: A Unified Information-Theoretic Analysis
arXiv:2506. 00849v2 Announce Type: replace Abstract: Despite the empirical success of Diffusion Models (DMs) and Variational Autoencoders (VAEs), their generalization performance remains theoretically underexplored, especially lacking a full consideration of the shared encoder-generator structure.
Related stories
arXiv Machine Learning
Aug 6
Regularization can make diffusion models more efficient
arXiv:2502. 09151v3 Announce Type: replace Abstract: Diffusion models are one of the key architectures of generative AI.
arXiv Machine Learning
5d ago
Information-Theoretic Scaling Laws for Generative Diffusion Training
The paper introduces an information-theoretic measure called model information to quantify how much a diffusion model’s outputs depend on its specific training dataset. By analyzing how this measure scales with model size and dataset size, the authors identify two distinct regimes: a nonlinear regime where memorization transitions to imperfect generalization, and a linear regime where generalization improves and sample quality rises. They propose a scaling law that captures these regimes and demonstrate that it can predict memorization and sample quality beyond the fitted data range.
arXiv Machine Learning
Oct 9
Barron Optimal Transport I: Generative Modeling
The paper introduces a new optimal transport framework that uses the Barron energy—an intrinsic neural network complexity measure—instead of the traditional kinetic energy. By modeling transport as deep residual networks that map a reference distribution to a target distribution, the authors define a new metric on probability measures and analyze its properties for generative modeling. They demonstrate that diffusion generative models are suboptimal in this Barron geometry and explore how adaptivity can improve alternative generative approaches.
arXiv AI
Jul 24
From Atoms to Entropy: Optimal Noise Allocation for Diffusion Training in the Convex Regime
arXiv:2607. 20540v1 Announce Type: cross Abstract: How should a diffusion model decide which noise levels to train on, and how much?
arXiv Machine Learning
Jun 10
The Emergence of Reproducibility and Generalizability in Diffusion Models
arXiv:2310. 05264v5 Announce Type: replace Abstract: In this work, we investigate an intriguing and prevalent phenomenon of diffusion models which we term as "consistent model reproducibility": given the same starting noise input and a deterministic sampler, different diffusion models often yield remarkably similar outputs.
arXiv Statistics ML
5d ago
Latent Nonlinear Denoising Score Matching for Enhanced Learning of Structured Distributions
The paper introduces Latent Nonlinear Denoising Score Matching (LNDSM), a method that integrates nonlinear, structure-informed latent diffusions with a variational autoencoder framework. LNDSM designs the forward diffusion to admit a chosen reference distribution, such as a Gaussian mixture, and derives a computable approximation to a cross‑entropy upper bound using Euler‑Maruyama discretization. Experiments on benchmark image datasets show that LNDSM improves sample quality, mode balance, and training efficiency compared to conventional latent score‑based generative models.
arXiv Machine Learning
Jul 22
ROMS-IMLE: A Minimalist Approach to Competitive Single-Step Generative Modelling
arXiv:2607. 19332v1 Announce Type: new Abstract: Generative models have undergone many generations of evolution, from VAEs/GANs to diffusion/flow matching.
arXiv Machine Learning
Jun 9
Self-Consistent Generative Paths via Admissible Random Variational Transport
arXiv:2606. 08953v1 Announce Type: new Abstract: Modern generative models often define an entire probability path from a simple prior to the data law, rather than only an endpoint map.
Hugging Face Trending Papers
Jun 22
PeLAP-A: Adaptive Latent Pruning for Lightweight Latent Diffusion Models
Latent diffusion models achieve strong generative performance by operating in a compressed latent space produced by a variational autoencoder (VAE). However, it remains unclear whether all latent channels contribute equally to the diffusion process, or whether significant redundancy exists.
arXiv AI
Jun 9
Unified Energy for Invariant and Independent Decoding in Diffusion Language Models
arXiv:2606. 09159v1 Announce Type: cross Abstract: Diffusion Language Models (DLMs) enable parallel text generation by iteratively denoising a full sequence, offering attractive flexibility compared to auto-regressive (AR) decoding.
arXiv AI
Jun 2
Ideas in Inference-time Scaling can Benefit Generative Pre-training Algorithms
arXiv:2503. 07154v3 Announce Type: replace-cross Abstract: Generative pre-training is often framed through a false dichotomy between autoregressive models for discrete signals and diffusion models for continuous signals.