← Back to all news
arXiv Machine Learning By Mo Zhou, Haoyang Ma, Rong Ge

Memorizing Long-tail Data Can Help Generalization Through Composition

Read the original on arXiv Machine Learning →

The paper investigates how memorizing long‑tail data can aid generalization when combined with a model’s ability to compose features. It demonstrates theoretically that, in a linear setting, memorization plus composition enables correct predictions on rare test examples that involve unseen combinations of long‑tail features. Experiments with neural networks on simple datasets confirm that this effect extends beyond linear models and that a model’s compositional capability depends on its architecture.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Sep 25

Pointwise Generalization in Deep Neural Networks

The paper introduces a pointwise generalization theory for fully connected deep neural networks, using a pointwise Riemannian Dimension derived from eigenvalues of learned feature representations across layers. This framework provides hypothesis-dependent, representation-aware generalization bounds that are significantly tighter than traditional size- or norm-based approaches, both theoretically and experimentally. The authors analytically identify structural properties that explain deep networks’ tractability and empirically show that the pointwise Riemannian Dimension captures feature compression, over‑parameterization effects, and optimizer bias.

By Shaojie Li, Yunbei Xu
safety
More like this →
Hugging Face Trending Papers
Jun 22

Sublinearly Structured Deep Neural Networks Achieve Feature Learning Consistency for Compositional Functions

Over the past decade, deep neural networks (DNNs) have achieved remarkable success on complex machine-learning tasks, yet the theoretical foundations of their performance remain incomplete. From a statistical viewpoint, a natural question is: can DNNs attain feature-learning and prediction consistency comparable to that of classical models?

computer-visionbenchmarks
More like this →