The paper investigates how a vanilla Transformer learns symmetries from finite data augmentation on point cloud datasets. It finds an ordering of learnability: non-angle-preserving symmetries are easiest, followed by angle-preserving symmetries, and finally base angle-preserving subgroups such as translation, rotation, and scale. The study also examines the Transformer's extrapolation behavior, performs a structural analysis of trained models to uncover interpretable mechanisms for invariance, and extends these findings to equivariant functions, suggesting that the identified mechanisms can serve as building blocks for learned equivariance.
arXiv:2608.31045v1 Announce Type: new Abstract: Rotational symmetry is one of the most important structural principles in machine learning on 3D data. In applications ranging from physics and materia...
arXiv:2510. 03511v3 Announce Type: replace-cross Abstract: While widespread, Transformers lack inductive biases for geometric symmetries common in science and computer vision.
arXiv:2608. 12010v1 Announce Type: new Abstract: Equivariant Neural Networks (ENNs) have empowered numerous applications in scientific fields.
Equivariant Neural Networks (ENNs) have empowered numerous applications in scientific fields. Despite their remarkable capacity for representing geometric structures, ENNs suffer from degraded expressivity when processing symmetric inputs: the output representations are invariant to transformations that extend beyond the input's symmetries.
arXiv:2512. 20043v3 Announce Type: replace Abstract: Symmetry is fundamental to understanding physical systems and can improve performance and sample efficiency in machine learning.
arXiv:2606. 24418v1 Announce Type: new Abstract: Data augmentation is a simple and model-agnostic approach for exploiting known invariances in learning problems.
arXiv:2207. 03116v4 Announce Type: replace Abstract: We introduce a general method for learning representations that are equivariant to symmetries of data.
arXiv:2606. 04108v1 Announce Type: cross Abstract: Single-view 3D generative models have achieved impressive visual quality, yet they are not designed to satisfy structural or functional requirements, and in practice, often fall short.
arXiv:2312. 08230v2 Announce Type: replace-cross Abstract: Detecting partial extrinsic symmetry in 3D geometry is a fundamental yet persistent challenge in computer vision and graphics, critical for tasks ranging from shape completion to procedural generation.
arXiv:2508. 07743v2 Announce Type: replace Abstract: While transformers excel in many settings, their application in the field of automated planning is limited.
arXiv:2602. 18948v2 Announce Type: replace Abstract: Transformer models contain substantial internal redundancy arising from coordinate-dependent representations and continuous symmetries, in model space and in head space, respectively.
arXiv:2509. 11218v2 Announce Type: replace-cross Abstract: Spatial transformations such as rotation and scale obscure the morphological cues needed for accurate image classification.