arXiv Machine Learning September 15, 2026 By Chon-Fai Kam, Miloud Bessafi, Frederic Cadet

Structured Features Overfit Where Random Features Grok

Read the original on arXiv Machine Learning →

arXiv:2609. 15047v1 Announce Type: new Abstract: Xu, Vardi and Safran (ICML 2026) prove that over-parameterized ridge regression over an unstructured random Gaussian feature map groks, with the delay between memorization and generalization growing as $1/\lambda$ in the weight decay.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Aug 31

Learning between the peaks: sharp asymptotics for kernel ridge regression under power-law anisotropy

arXiv:2608. 28564v1 Announce Type: cross Abstract: We study kernel ridge regression under anisotropic Gaussian data, where the input covariance decays as a power law with exponent $\alpha\geq 0$ for polynomial inner-product kernels.

By Lorenzo Rizzi, Arie Wortsman Zurich, Bruno Loureiro

safety

arXiv AI

Jul 28

Scale Weight Decay and Train Better

arXiv:2607. 23777v1 Announce Type: cross Abstract: The discovery of scaling laws has motivated training neural networks on ever increasing quantities of data.

By Anuj Apte

safety

arXiv AI

Jun 2

UR-JEPA: Uniform Rectifiability as a Regularizer for Joint-Embedding Predictive Architectures

arXiv:2606. 01443v1 Announce Type: cross Abstract: A central difficulty in training Joint-Embedding Predictive Architectures (JEPAs) is preventing representation collapse.

By Triet M. Le

rag

arXiv Machine Learning

Jul 17

To Grok Grokking: Provable Grokking in Ridge Regression

arXiv:2601. 19791v4 Announce Type: replace Abstract: We study grokking, the onset of generalization long after overfitting, in a classical ridge regression setting.

By Mingyue Xu, Gal Vardi, Itay Safran

arXiv AI

Jun 4

Low-Rank Decay for Grokking in Scale-Invariant Transformers: A Spectral-Geometric View

arXiv:2606. 04405v1 Announce Type: cross Abstract: Modern Transformer architectures frequently employ normalization mechanisms such as RMSNorm and Query-Key Normalization, making parts of the model approximately scale-invariant with respect to weight magnitudes.

By Mingyu Li

llms

arXiv AI

Aug 10

Post-Grokking Collapse at the Representation-Readout Interface in Muon-Trained Transformers

arXiv:2608. 07436v1 Announce Type: new Abstract: Under the standard split, Muon gets hidden matrices and AdamW embeddings/output head.

By Ali Janati, Kaoutar El Maghraoui, Andrei Kanavalau, Anass Belfatmi

llms rag safety

One email a morning, machine-written

Related stories

Learning between the peaks: sharp asymptotics for kernel ridge regression under power-law anisotropy

Scale Weight Decay and Train Better

UR-JEPA: Uniform Rectifiability as a Regularizer for Joint-Embedding Predictive Architectures

To Grok Grokking: Provable Grokking in Ridge Regression

Low-Rank Decay for Grokking in Scale-Invariant Transformers: A Spectral-Geometric View

Post-Grokking Collapse at the Representation-Readout Interface in Muon-Trained Transformers