← Back to all news
arXiv Statistics ML

Enforcing Symmetries on Pre-trained ML Models for Physics

The paper investigates post‑hoc symmetry enforcement on pre‑trained machine‑learning models used for physics simulations. For finite, compact groups, simple group averaging (the Reynolds operator) projects predictors onto invariant or equivariant functions with non‑increasing risk guarantees, and numerical experiments confirm its effectiveness. The authors extend the framework to non‑compact groups, proposing three alternatives—Weyl unitary trick, data‑dependent orthogonal projection, and frame averaging—each evaluated theoretically and empirically on the unit‑rescaling and Lorentz groups; results show that only the orthogonal projection consistently improves performance, while the other methods help mainly in out‑of‑distribution or low‑data scenarios, and that training directly on invariant features can match or surpass post‑hoc projections.

Related stories

arXiv AI
Oct 15

Exact Symmetry as Algebra: Group-Structured Tensors that Enforce Physical Selection Rules

The paper introduces a tensor algebra, ∗_G, that enforces exact symmetry in machine‑learned models of molecular and crystal responses. By applying this algebra to the Orb‑v3 force field, the authors demonstrate that many models incorrectly alter a crystal’s space group during relaxation, and they show how projections derived from ∗_G can correct these errors without retraining. The method yields reference‑free error bounds and symmetry‑preserving low‑rank approximations, reducing space‑group changes to at most 2%.

By Paulina Hoyos, Shashanka Ubaru, Dongsung Huh, Vasileios Kalantzis, Kenneth L. Clarkson, Misha Kilmer, Haim Avron, Lior Horesh
More like this →
arXiv AI
Jun 3

Exact equivariance, kept through training, buys zero-shot generalisation across the symmetry group

arXiv:2606. 03003v1 Announce Type: cross Abstract: A latent world model built from an equivariant encoder $E$ and an equivariant predictor $f$ inherits a provable symmetry of its training loss: when the world's dynamics genuinely carries a group $G$ acting on latents by an orthogonal representation $\rho(g)$, the one-step prediction relMSE is exactly invariant across the whole group, so fitting the dynamics on a restricted slice of orientations mathematically determines it on the entire orbit (j\v{u} y\=i f\v{a}n s\=an).

By Hongbo Wang (Stony Brook University)
More like this →
arXiv AI
Jun 30

Representation Learning for Equivariant Inference with Guarantees

arXiv:2505. 19809v3 Announce Type: replace-cross Abstract: In many real-world applications of regression, conditional probability estimation, and uncertainty quantification, exploiting symmetries rooted in physics or geometry can dramatically improve generalization and sample efficiency.

By Daniel Ordo\~nez-Apraez, Vladimir Kosti\'c, Alek Fr\"ohlich, Vivien Brandt, Karim Lounici, Massimiliano Pontil
robotics
More like this →
arXiv Machine Learning
Oct 8

How Do Transformers Learn to Represent Symmetries?

The paper investigates how a vanilla Transformer learns symmetries from finite data augmentation on point cloud datasets. It finds an ordering of learnability: non-angle-preserving symmetries are easiest, followed by angle-preserving symmetries, and finally base angle-preserving subgroups such as translation, rotation, and scale. The study also examines the Transformer's extrapolation behavior, performs a structural analysis of trained models to uncover interpretable mechanisms for invariance, and extends these findings to equivariant functions, suggesting that the identified mechanisms can serve as building blocks for learned equivariance.

By Eduardo Santos-Escriche, Valerie Engelmayer, Ya-Wei Eileen Lin, Stefanie Jegelka
llms
More like this →
Hugging Face Trending Papers
Aug 12

Reducing Symmetry Increase in Equivariant Neural Networks

Equivariant Neural Networks (ENNs) have empowered numerous applications in scientific fields. Despite their remarkable capacity for representing geometric structures, ENNs suffer from degraded expressivity when processing symmetric inputs: the output representations are invariant to transformations that extend beyond the input's symmetries.

More like this →
arXiv Machine Learning
Jul 30

Exact Symmetry as Algebra: A Machine-Verified Tensor Calculus that Enforces Physical Selection Rules

arXiv:2605. 20440v2 Announce Type: replace Abstract: Symmetry is central to the physical sciences, yet machine learning usually captures it only approximately, leaving a residual per-step equivariance error $\varepsilon$ that compounds with depth $M$ as $M\varepsilon$, whereas exact equivariance holds at unbounded depth; we demonstrate this divergence at fourteen orders of magnitude.

By Paulina Hoyos, Shashanka Ubaru, Dongsung Huh, Vasileios Kalantzis, Kenneth L. Clarkson, Misha Kilmer, Haim Avron, Lior Horesh
More like this →