arXiv:2510. 22778v3 Announce Type: replace-cross Abstract: We develop a free-probabilistic framework for denoising diffusion, in which the data is a self-adjoint operator and its law a spectral distribution.
arXiv:2606. 13796v1 Announce Type: cross Abstract: Recursive training of generative models on their own outputs can lead to model collapse, a compounding drift away from the true data distribution.
The article "Foundations of Diffusion Models in General State Spaces: A Self-Contained Introduction" presents a unified primer on diffusion models that applies to both continuous Euclidean data and discrete categorical structures. It develops discrete-time forward noising via Markov kernels and learned reverse dynamics, and connects these to continuous-time limits such as stochastic differential equations in ρ^d and continuous-time Markov chains on finite alphabets, deriving the corresponding Fokker–Planck and master equations. The work also shows how different forward corruption choices—Gaussian processes for continuous spaces and structured categorical transition kernels for discrete spaces—affect reverse dynamics and the evidence lower bound used in training, offering a layered exposition for newcomers, practitioners, and experts alike.
arXiv:2602. 02908v2 Announce Type: replace-cross Abstract: Diffusion models trained on different, non-overlapping subsets of a dataset often produce strikingly similar outputs when given the same noise seed.
arXiv:2605. 17232v2 Announce Type: replace Abstract: Discrete diffusion has become a leading framework for generative modeling in various applications including language, vision, and biology.
arXiv:2610. 12052v1 Announce Type: cross Abstract: Despite their empirical success, why diffusion models overcome the bottlenecks of classical score-based samplers remains unclear.
arXiv:2607. 07967v1 Announce Type: cross Abstract: Diffusion-based policies have recently emerged as powerful policy parameterizations for reinforcement learning, representing state-conditioned action distributions as terminal laws of diffusion processes with parameterized drifts.
The paper introduces a Projected Riemannian Gradient Descent (RGD) algorithm for computing the Bures‑Wasserstein barycenter of positive definite matrices, achieving dimension‑independent linear convergence at unit step size. It resolves a previous dichotomy by showing that clipping eigenvalues to a fixed interval yields a closed‑form, non‑expansive projection in the BW metric, allowing the algorithm to match the empirical speed of unit‑step RGD while maintaining theoretical guarantees. The method also extends to the invariant matrix projection problem, providing a unified dimension‑independent analysis.
The paper investigates how spectral summaries and scalar learning curves can reveal properties of centered quadratic networks. By constructing analytic positive input densities with Gaussian one- and two-coordinate marginals, the authors show that certain teachers with a nonzero cycle invariant produce distinct factorized loss curves, while others yield identical anchored spectral measures. The study further explores the sample complexity required for noisy scalar loss queries and bounded initialization interventions, identifies conditions under which a single scalar loss curve can recover the initial predictor and teacher spectrum, and provides exact recovery guarantees for fixed dimensions and distinct teacher sums.
We study what spectral summaries and scalar learning curves reveal about centered quadratic networks. We construct analytic positive input densities with exactly Gaussian one- and two-coordinate margi...
arXiv:2605. 17232v5 Announce Type: replace Abstract: Discrete diffusion has become a leading framework for generative modeling in various applications including language, vision, and biology.
The paper analyzes the stochastic optimizer Muon, which orthogonalizes an exponential moving average of minibatch gradients, and compares its behavior to the baseline SignSGD on a high‑dimensional matrix‑valued least squares problem. It derives deterministic dynamics showing Muon’s sensitivity to an effective batch size and its failure to resolve small eigenmodes when whitening is exact and the effective batch is small, while Momentum and finite Newton–Shulz iterations can mitigate this. The study presents a phase diagram in terms of data and target exponents, indicating when Muon or SignSGD is preferable based on covariance spectrum steepness and target loss scaling.