← Back to all news
arXiv Machine Learning By Aditya Cowsik, Adithya Sriram

Hierarchical Prototype Emergence in Modern Hopfield Models

Read the original on arXiv Machine Learning →

The paper studies how hierarchical correlations in data can be learned by a dense Hopfield network with polynomial activation. It analytically derives conditions for each level of a hierarchical memory model to be locally stable, meaning they correspond to local energy minima. Using prototype reconstruction as a minimal generalization test, the authors show that only a quasi‑polynomial amount of information is needed to generalize beyond specific memories or groups, and they observe a similar phase diagram for Fashion‑MNIST data.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Jul 22

Free energy landscape of Dense Associative Memory

arXiv:2607. 19195v1 Announce Type: cross Abstract: Using large deviations theory, we solve and obtain a general expression for the free energy functional for a broad class of associative memories, including dense associative memories.

By Sumedha, Abhishek Singh
More like this →
arXiv Machine Learning
Sep 14

Diffusion Models and Concept Formation

The paper argues that diffusion models, originally developed for image synthesis, implicitly perform concept formation similar to the Cobweb cognitive model. Both models build hierarchical density structures using Gaussian prototypes, treat categorization as score‑following to reduce uncertainty, and exhibit a basic level of abstraction. The authors demonstrate this correspondence by extracting a diffusion hierarchy from MNIST and Fashion‑MNIST data and comparing its basic level to that of Cobweb, suggesting diffusion models can serve as a continuous, scalable instantiation of concept formation.

By Zekun Wang, Karthik Singaravadivelan, Christopher J. MacLellan
diffusion
More like this →
arXiv AI
Sep 3

Language Diffusion Models are Associative Memories Capable of Retrieving Unseen Data

The paper investigates when language diffusion models, specifically Uniform-based Discrete Diffusion Models (UDDMs), shift from memorizing training data to generalizing to new data. It shows that UDDMs act as associative memories, forming basins of attraction around stored examples without requiring an explicit energy function. By measuring token recovery and conditional entropy, the authors identify a sharp transition governed by training set size, where memorization (vanishing entropy) gives way to generalization (finite entropy).

By Bao Pham, Mohammed J. Zaki, Luca Ambrogioni, Dmitry Krotov, Matteo Negri
diffusion
More like this →