arXiv Machine Learning
Jun 24
Data Augmentation: A Fourier Analysis Perspective
arXiv:2606. 24418v1 Announce Type: new Abstract: Data augmentation is a simple and model-agnostic approach for exploiting known invariances in learning problems.
Breaking Data Symmetry is Needed For Generalization in Feature Learning Kernels
arXiv:2604. 00316v2 Announce Type: replace-cross Abstract: Grokking occurs when a model achieves high training accuracy but generalization to unseen test points happens long after that.
Related stories
arXiv Machine Learning
Sep 10
Efficient Learning and Symmetry Discovery under Exact Invariances
arXiv:2609.07031v1 Announce Type: new Abstract: Learning with group invariances is central to many scientific and geometric learning problems, yet its computational foundations remain poorly understo...
arXiv AI
Jul 2
K-Inverse-RFM: A Modified RFM that Bridges the Gap to Neural Networks for Data-Corrupted Mathematical Tasks
arXiv:2607. 00329v1 Announce Type: cross Abstract: Recursive Feature Machines (RFMs) are a class of kernel machines that utilize the Average Gradient Outer Product (AGOP) as a mechanism for feature learning.
arXiv Machine Learning
Aug 10
Equivariant Sparse Autoencoders: Mechanistic Interpretability of Neural Networks on Symmetric Data
arXiv:2511. 09432v2 Announce Type: replace Abstract: Machine learning (ML) models achieve remarkable performance but remain hard to interpret due to their scale and complexity.
Hugging Face Trending Papers
Aug 12
Reducing Symmetry Increase in Equivariant Neural Networks
Equivariant Neural Networks (ENNs) have empowered numerous applications in scientific fields. Despite their remarkable capacity for representing geometric structures, ENNs suffer from degraded expressivity when processing symmetric inputs: the output representations are invariant to transformations that extend beyond the input's symmetries.
arXiv Machine Learning
Aug 13
Reducing Symmetry Increase in Equivariant Neural Networks
arXiv:2608. 12010v1 Announce Type: new Abstract: Equivariant Neural Networks (ENNs) have empowered numerous applications in scientific fields.
arXiv Machine Learning
Jul 17
To Grok Grokking: Provable Grokking in Ridge Regression
arXiv:2601. 19791v4 Announce Type: replace Abstract: We study grokking, the onset of generalization long after overfitting, in a classical ridge regression setting.
arXiv Machine Learning
Sep 16
Universal Feature Selection with Noisy Observations and Weak Symmetry Conditions
arXiv:2605.09396v2 Announce Type: replace-cross Abstract: This paper relaxes the restrictive symmetry conditions adopted in [4], [5] and extends their universal feature selection framework to accommo...
arXiv Machine Learning
Sep 23
A Spectral Theory of Grokking: Weight Decay induces Feature Learning
The paper presents a spectral theory explaining the phenomenon of grokking, where an initial fit to training data is followed by a delayed improvement in generalization. It shows that for homogeneous networks trained with squared loss and L₂ weight decay, residuals after memorization influence the neural tangent kernel (NTK) dynamics, leading to a transition from lazy to rich learning. The theory predicts that grokking timescales depend on the product of learning rate and weight decay, and that stronger decay can halt fitting, with empirical validation on modular addition tasks using MLPs and Transformers.
arXiv Machine Learning
Jul 3
On the Asymptotics of Self-Supervised Pre-training: Two-Stage M-Estimation and Representation Symmetry
arXiv:2603. 27631v2 Announce Type: replace Abstract: Self-supervised pre-training, where large corpora of unlabeled data are used to learn representations for downstream fine-tuning, has become a cornerstone of modern machine learning.
arXiv AI
Sep 15
Dissociating performance from compositional feature learning
arXiv:2505.09716v3 Announce Type: replace-cross Abstract: Out-of-distribution (OOD) generalisation is considered a hallmark of human and animal intelligence. To achieve OOD through composition, a sys...
arXiv Machine Learning
Sep 25
Pointwise Generalization in Deep Neural Networks
The paper introduces a pointwise generalization theory for fully connected deep neural networks, using a pointwise Riemannian Dimension derived from eigenvalues of learned feature representations across layers. This framework provides hypothesis-dependent, representation-aware generalization bounds that are significantly tighter than traditional size- or norm-based approaches, both theoretically and experimentally. The authors analytically identify structural properties that explain deep networks’ tractability and empirically show that the pointwise Riemannian Dimension captures feature compression, over‑parameterization effects, and optimizer bias.