arXiv:2608. 16664v1 Announce Type: cross Abstract: We study the Random Quadratic Form (RQF) on a sphere in the presence of random Brownian forcing.
arXiv:2603. 06187v2 Announce Type: replace-cross Abstract: We introduce the Random Quadratic Form (RQF): a stochastic differential equation which formally corresponds to the gradient flow of a random quadratic functional on a sphere.
The paper investigates why diffusion models sometimes fail to learn symmetries present in data distributions, such as different orientations of a dog. Focusing on linear diffusion models, it identifies three possible sources of symmetry breaking—non‑symmetric initial weights, noisy learning targets, and finite‑sample noise—and shows that target noise is the most robust cause, persisting even with abundant data. Using random matrix theory, the authors derive formulas for average symmetry error, revealing that smaller symmetries are harder to learn.
The paper studies momentum-based optimization when updates are sparse, focusing on two tractable models: a least‑squares problem with sparse inputs and a logistic regression problem with a rare class. It derives exact closed‑form second‑moment dynamics and analyzes their high‑dimensional limits across three scaling exponents for sparsity, batch size, and momentum decay. The resulting phase diagram is governed by the ratio of a momentum retention timescale and a learning timescale, revealing regimes where the dynamics match SGD, become unstable, or recover classical heavy‑ball behavior, and showing that oscillatory behavior depends on token sparsity, leading to a spectral conflict for global momentum.
arXiv:2607. 04135v1 Announce Type: cross Abstract: The remarkable ability of modern neural networks to generalize improves with increasing network capacity, even when the number of model parameters or effective degrees of freedom exceeds the number of training data points.
The paper demonstrates that long‑time stochastic dynamics can be predicted using generative diffusion models trained only on configuration pairs separated by a short, fixed time lag, without requiring the underlying equations of motion. Applied to two‑dimensional Model B and driven colloids in a periodic optical potential, the learned transition kernels accurately reproduce dynamic critical scaling, self‑similar coarsening, and experimental observables such as particle current and mean passage time, even on larger lattices and unseen initial conditions. This shows that short‑time observations contain sufficient information to forecast emergent non‑equilibrium behavior over extended periods.
The paper investigates how deep residual networks behave when their initial weights are correlated across layers. It confirms a conjecture that such correlated initializations interpolate between a Brownian stochastic differential equation (for independent weights) and an ordinary differential equation (for perfectly correlated weights). By applying a feature function to a stationary Gaussian sequence with regularly varying correlation, the authors prove that a unique critical scaling exists, leading the infinite‑depth limit to a Young differential equation driven by a Hermite process, which reduces to fractional Brownian motion when the feature function has Hermite rank one. The study shows that the correlation structure and Hermite rank of the initialization uniquely determine the critical scaling and asymptotic limit, making them meaningful hyperparameters in the asymptotic regime, whereas finite‑variance i.i.d. initialization always yields a Brownian driver regardless of distribution.
arXiv:2505. 19619v3 Announce Type: replace Abstract: Deep generative models have recently garnered significant attention across various fields, from physics to chemistry, where sampling from unnormalized Boltzmann-like distributions represents a fundamental challenge.
The remarkable ability of modern neural networks to generalize improves with increasing network capacity, even when the number of model parameters or effective degrees of freedom exceeds the number of training data points. This phenomenon is all the more surprising given that generalization error diverges when the number of model parameters approaches a critical value from below.
arXiv:2608. 01582v1 Announce Type: cross Abstract: Symmetry is central to modern machine learning and physics: invariances and equivariances improve sample efficiency, robustness, and out-of-distribution generalization, while symmetry principles guide scientific modeling.
arXiv:2608.25606v2 Announce Type: replace-cross Abstract: The simple exclusion process (SEP) is a paradigmatic model for nonequilibrium transport, yet its rich dynamics over an exponentially large co...
The paper investigates how finite‑stepsize stochastic gradient descent (SGD) breaks scale symmetry in undercomplete linear autoencoders. It shows that SGD induces a directed scale drift that favors large decoder weights on the PCA solution manifold, a process that is slow and analytically tractable but eventually halted by a stability boundary. The resulting solutions exhibit sharper loss Hessians compared to a balanced baseline, illustrating how loss geometry can transform residual gradient noise into directed motion along equivalent solution manifolds.
arXiv:2605. 26814v2 Announce Type: replace-cross Abstract: We train a pair of autoregressive models to construct zero-mean control variates to mitigate the sign problem in quantum Monte Carlo simulations.