arXiv Machine Learning
Jun 2
Theoretical Analysis of Engression and Reverse Markov Engression
arXiv:2606. 01002v1 Announce Type: cross Abstract: Engression is a recently proposed and effective framework for conditional distribution learning.
Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks
The paper presents a theoretical framework for approximating ratio-type functionals that arise in conditional generative modeling, specifically when the target density is expressed as a ratio of two kernel-based marginal densities. It proves that deep neural networks using the SignReLU activation can approximate these ratios with established L^p(Omega) bounds and convergence rates under standard regularity assumptions. Applying the framework to Denoising Diffusion Probabilistic Models, the authors construct a SignReLU-based estimator for the reverse process and derive bounds on the excess Kullback–Leibler risk, decomposing it into approximation and estimation errors to provide generalization guarantees for finite-sample training.
Related stories
arXiv Machine Learning
Aug 26
Generalization, memorization, and overfitting for diffusion models trained in the lazy high-dimensional regime
The paper investigates diffusion models trained in a lazy high‑dimensional regime, extending benign overfitting theory to generative settings. By analyzing denoising score matching in a vector‑valued RKHS with an inner‑product kernel, the authors derive exact risk trajectories under gradient flow when the number of samples scales proportionally with dimensionality. These trajectories reveal three distinct phases—spectral generalization, noise‑dominated interpolation, and empirical Bayes memorization—whose interplay shapes the distribution of generated samples.
arXiv Machine Learning
Jul 27
From Score Approximation to Distribution Approximation in Score-Based Diffusion Models
arXiv:2607. 22199v1 Announce Type: new Abstract: Score-based diffusion models have achieved remarkable empirical success in generative modeling, yet their approximation-theoretic foundations remain incomplete.
arXiv Machine Learning
Jun 8
Generalization of Diffusion Models Arises with a Balanced Representation Space
arXiv:2512. 20963v3 Announce Type: replace Abstract: Diffusion models excel at generating high-quality, diverse samples, yet they risk memorizing training data when overfit to the training objective.
arXiv Machine Learning
Jul 28
From Score Learning to Discretized Sampling: An End-to-End Generalization Analysis of Diffusion Models
arXiv:2607. 23226v1 Announce Type: new Abstract: Despite the empirical success of score-based diffusion models, a complete theoretical understanding of how finite-sample learning, network parameterization, and numerical discretization jointly dictate generative quality remains underdeveloped.
arXiv Machine Learning
Jul 7
Non-asymptotic Convergence of Stochastic Gradient Descent in Score-based Generative Models
arXiv:2607. 04775v1 Announce Type: cross Abstract: Score-based Generative Models (SGMs) have achieved impressive performance in data generation across a wide range of applications.
arXiv AI
Jun 4
Markov Chain Decoders Overcome the Heavy-Tail Limitations of Lipschitz Generative Models
arXiv:2605. 18931v2 Announce Type: replace-cross Abstract: Heavy-tailed distributions are prevalent in performance evaluation, network traffic, and risk modeling.
arXiv Machine Learning
Aug 17
Offline Deep Q* Estimation with Diffusion Models
arXiv:2608. 14401v1 Announce Type: cross Abstract: In offline RL, estimating the optimal action-value function $Q^*$ can be formulated as solving the optimal Bellman equation based solely on offline observations.
arXiv Machine Learning
Jun 16
Brownian Kernel Ladders
arXiv:2606. 15812v1 Announce Type: new Abstract: Constructing mathematically tractable function spaces that capture hierarchical compositional representations remains a central challenge in statistical learning theory.
Hugging Face Trending Papers
Jul 6
Non-asymptotic Convergence of Stochastic Gradient Descent in Score-based Generative Models
Score-based Generative Models (SGMs) have achieved impressive performance in data generation across a wide range of applications. While the statistical properties of their sampling procedures are increasingly well understood, the optimization dynamics underlying their training remain less explored.
arXiv Machine Learning
Jul 21
Semi-Supervised Conditional Generative Learning through Stochastic Interpolation and Sufficient Representations
arXiv:2607. 16725v1 Announce Type: cross Abstract: Conditional generative modeling remains a challenging problem in semi-supervised settings where labeled data is scarce but unlabeled samples are abundant.
arXiv AI
Jun 2
Ideas in Inference-time Scaling can Benefit Generative Pre-training Algorithms
arXiv:2503. 07154v3 Announce Type: replace-cross Abstract: Generative pre-training is often framed through a false dichotomy between autoregressive models for discrete signals and diffusion models for continuous signals.