← Back to all news
arXiv Machine Learning By Sahel Torkamani, Henry Gouk, Rik Sarkar

Generalization in Neural Networks Through the Lens of Magnitude Potential

Read the original on arXiv Machine Learning →

The paper introduces magnitude potential, a metric derived from metric magnitude theory, to assess how well a point is represented by a set. By computing the ratio of magnitude potential relative to a class versus the entire dataset at the logit layer, the authors find correlations with Feldman memorization scores and detect structural changes in decision boundaries, including grokking in modular arithmetic. This ratio remains informative even when neural collapse is suppressed, highlighting its robustness in capturing intra‑ and inter‑class geometric structure.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Sep 25

Pointwise Generalization in Deep Neural Networks

The paper introduces a pointwise generalization theory for fully connected deep neural networks, using a pointwise Riemannian Dimension derived from eigenvalues of learned feature representations across layers. This framework provides hypothesis-dependent, representation-aware generalization bounds that are significantly tighter than traditional size- or norm-based approaches, both theoretically and experimentally. The authors analytically identify structural properties that explain deep networks’ tractability and empirically show that the pointwise Riemannian Dimension captures feature compression, over‑parameterization effects, and optimizer bias.

By Shaojie Li, Yunbei Xu
safety
More like this →