The paper introduces a generalized score matching objective for parameter estimation on convex subsets of ρ^d, derived from Minimum Probability Flow learning. It shows that this objective is a proper local scoring rule of second order, ensuring recovery of the true density when minimized, and proves convexity and consistency for exponential family models under standard conditions. Experiments demonstrate the method’s effectiveness on constrained domains where the partition function is intractable, including a generative modeling use‑case.
By Nishanth Shetty, Saisuchith Mahajan, Chandra Sekhar Seelamantula
arXiv:2606. 19876v1 Announce Type: new Abstract: The score matching problem is a central training objective in modern generative modeling, diffusion models, fitting unnormalized statistical models, and inverse problems.
By Alexander Tyurin
arXiv:2508. 01597v2 Announce Type: replace Abstract: Score Matching (SM) is a powerful framework for estimating the log-density derivatives of a distribution without calculating its normalizing constants.
By Juyan Zhang, Rhys Newbury, Xinyang Zhang, Tin Tran, Dana Kulic, Michael Burke
arXiv:2607. 15485v1 Announce Type: new Abstract: Score-based generative models exhibit a puzzling behavior: they often appear to cover all modes of a target multimodal distribution and yet may fail to learn the correct relative mode amplitudes, which can be interpreted as mixture weights.
By Andrew Dennehy, Ramchandran Muthukumar, Rebecca Willett, Nisha Chandramoorthy
arXiv:2608. 19067v1 Announce Type: cross Abstract: The empirical success of diffusion models in generative modelling has motivated theoretical work, including quantitative error bounds and qualitative analyses that characterise the different phases of denoising.
By Yuga Iguchi, Paul Fearnhead
arXiv:2510. 12744v2 Announce Type: replace-cross Abstract: We develop a unified statistical framework for softmax-gated Gaussian mixture of experts (SGMoE) that addresses three long-standing obstacles in parameter estimation and model selection: (i) non-identifiability of gating parameters up to common translations, (ii) intrinsic gate-expert interactions that induce coupled differential relations in the likelihood, and (iii) the tight numerator-denominator coupling in the softmax-induced conditional density.
By Do Tien Hai, Trung Nguyen Mai, TrungTin Nguyen, Nhat Ho, Binh T. Nguyen, Christopher Drovandi
The paper introduces an amortized learning framework for selecting bandwidths in kernel density estimation by optimizing the logarithmic score across a distribution of tasks. It uses a truncated-and-renormalized bounded-support formulation and affine standardization to achieve stable learning and transferability across different intervals. Experiments on Gaussian samples, a multi-family benchmark, and randomized Gaussian mixtures demonstrate that the learned selector outperforms traditional methods such as Silverman’s rule, Sheather–Jones, and least‑squares cross‑validation, especially for small or heterogeneous samples.
By Junyi Liang, Hailiang Du
arXiv:2607. 04442v1 Announce Type: cross Abstract: Diffusion models (DMs) are a state-of-the-art generative method to approximately sample from an unknown distribution.
By Benjamin Dupuis, Tyler Farghly, Maxime Haddouche, Alain Durmus, Umut Simsekli
arXiv:2608.20635v1 Announce Type: cross
Abstract: Discrete diffusion models have demonstrated strong performance across a range of datasets, including natural language data and graph-structured data....
By Cholyeon Cho, Yuchen Wu
arXiv:2501. 12982v3 Announce Type: replace-cross Abstract: This paper investigates how diffusion generative models leverage (unknown) low-dimensional structure to accelerate sampling.
By Jiadong Liang, Zhihan Huang, Yuxin Chen
arXiv:2606. 11469v1 Announce Type: cross Abstract: We study the task of density estimation, where we hope to accurately estimate a probability density from $n$ samples.
By Spencer Compton, Jerry Li
arXiv:2606. 06179v1 Announce Type: cross Abstract: Score-based diffusion models are typically trained by minimizing the $L^2$ score matching error, and standard theoretical analyses rely on this quantity to bound the sampling discrepancy between the learned and target distributions.
By Na\"il B. Khelifa, Richard E. Turner, Ramji Venkataramanan