arXiv:2608. 16664v1 Announce Type: cross Abstract: We study the Random Quadratic Form (RQF) on a sphere in the presence of random Brownian forcing.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.
arXiv:2603. 06187v2 Announce Type: replace-cross Abstract: We introduce the Random Quadratic Form (RQF): a stochastic differential equation which formally corresponds to the gradient flow of a random quadratic functional on a sphere.
The paper investigates why diffusion models sometimes fail to learn symmetries present in data distributions, such as different orientations of a dog. Focusing on linear diffusion models, it identifies three possible sources of symmetry breaking—non‑symmetric initial weights, noisy learning targets, and finite‑sample noise—and shows that target noise is the most robust cause, persisting even with abundant data. Using random matrix theory, the authors derive formulas for average symmetry error, revealing that smaller symmetries are harder to learn.
The paper studies momentum-based optimization when updates are sparse, focusing on two tractable models: a least‑squares problem with sparse inputs and a logistic regression problem with a rare class. It derives exact closed‑form second‑moment dynamics and analyzes their high‑dimensional limits across three scaling exponents for sparsity, batch size, and momentum decay. The resulting phase diagram is governed by the ratio of a momentum retention timescale and a learning timescale, revealing regimes where the dynamics match SGD, become unstable, or recover classical heavy‑ball behavior, and showing that oscillatory behavior depends on token sparsity, leading to a spectral conflict for global momentum.
arXiv:2607. 04135v1 Announce Type: cross Abstract: The remarkable ability of modern neural networks to generalize improves with increasing network capacity, even when the number of model parameters or effective degrees of freedom exceeds the number of training data points.
The paper demonstrates that long‑time stochastic dynamics can be predicted using generative diffusion models trained only on configuration pairs separated by a short, fixed time lag, without requiring the underlying equations of motion. Applied to two‑dimensional Model B and driven colloids in a periodic optical potential, the learned transition kernels accurately reproduce dynamic critical scaling, self‑similar coarsening, and experimental observables such as particle current and mean passage time, even on larger lattices and unseen initial conditions. This shows that short‑time observations contain sufficient information to forecast emergent non‑equilibrium behavior over extended periods.
The paper investigates how deep residual networks behave when their initial weights are correlated across layers. It confirms a conjecture that such correlated initializations interpolate between a Brownian stochastic differential equation (for independent weights) and an ordinary differential equation (for perfectly correlated weights). By applying a feature function to a stationary Gaussian sequence with regularly varying correlation, the authors prove that a unique critical scaling exists, leading the infinite‑depth limit to a Young differential equation driven by a Hermite process, which reduces to fractional Brownian motion when the feature function has Hermite rank one. The study shows that the correlation structure and Hermite rank of the initialization uniquely determine the critical scaling and asymptotic limit, making them meaningful hyperparameters in the asymptotic regime, whereas finite‑variance i.i.d. initialization always yields a Brownian driver regardless of distribution.