← Back to all news
arXiv Machine Learning By John J. Vastola, Kanaka Rajan

Quantifying the failure of symmetry learning in linear diffusion models due to initialization and finite-sample effects

Read the original on arXiv Machine Learning →

The paper investigates why diffusion models sometimes fail to learn symmetries present in data distributions, such as different orientations of a dog. Focusing on linear diffusion models, it identifies three possible sources of symmetry breaking—non‑symmetric initial weights, noisy learning targets, and finite‑sample noise—and shows that target noise is the most robust cause, persisting even with abundant data. Using random matrix theory, the authors derive formulas for average symmetry error, revealing that smaller symmetries are harder to learn.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
3d ago

Information-Theoretic Scaling Laws for Generative Diffusion Training

The paper introduces an information-theoretic measure called model information to quantify how much a diffusion model’s outputs depend on its specific training dataset. By analyzing how this measure scales with model size and dataset size, the authors identify two distinct regimes: a nonlinear regime where memorization transitions to imperfect generalization, and a linear regime where generalization improves and sample quality rises. They propose a scaling law that captures these regimes and demonstrate that it can predict memorization and sample quality beyond the fitted data range.

By Druv Pai, Dejan Stancevic, Yi Ma, Luca Ambrogioni
diffusion
More like this →
arXiv Statistics ML
3d ago

Symmetry Breaking Is the Price of Finite Capacity

The paper investigates how limited model capacity forces symmetry breaking in problems that are inherently rotation‑invariant. It shows that when a model’s capacity cuts across a symmetry block, the optimal solution must abandon symmetry, and quantifies the exact loss incurred for a Gaussian bilinear predictor. Experiments on attention heads confirm that the predicted symmetry‑breaking behavior occurs at the expected head dimensions and rotation sensitivities.

By Maggie Chlon, Leon Chlon
More like this →
arXiv Machine Learning
Jun 2

Symmetries in PAC-Bayesian Learning

arXiv:2510. 17303v2 Announce Type: replace Abstract: Symmetries are known to improve the empirical performance of machine learning models, yet theoretical guarantees explaining these gains remain limited.

By Armin Beck, Peter Ochs
More like this →