The paper studies how hierarchical correlations in data can be learned by a dense Hopfield network with polynomial activation. It analytically derives conditions for each level of a hierarchical memory model to be locally stable, meaning they correspond to local energy minima. Using prototype reconstruction as a minimal generalization test, the authors show that only a quasi‑polynomial amount of information is needed to generalize beyond specific memories or groups, and they observe a similar phase diagram for Fashion‑MNIST data.
arXiv:2607. 19195v1 Announce Type: cross Abstract: Using large deviations theory, we solve and obtain a general expression for the free energy functional for a broad class of associative memories, including dense associative memories.
The paper argues that diffusion models, originally developed for image synthesis, implicitly perform concept formation similar to the Cobweb cognitive model. Both models build hierarchical density structures using Gaussian prototypes, treat categorization as score‑following to reduce uncertainty, and exhibit a basic level of abstraction. The authors demonstrate this correspondence by extracting a diffusion hierarchy from MNIST and Fashion‑MNIST data and comparing its basic level to that of Cobweb, suggesting diffusion models can serve as a continuous, scalable instantiation of concept formation.
arXiv:2511. 02584v2 Announce Type: replace-cross Abstract: Associative memory, traditionally modeled by Hopfield networks, enables the retrieval of previously stored patterns from partial or noisy cues.
The paper investigates when language diffusion models, specifically Uniform-based Discrete Diffusion Models (UDDMs), shift from memorizing training data to generalizing to new data. It shows that UDDMs act as associative memories, forming basins of attraction around stored examples without requiring an explicit energy function. By measuring token recovery and conditional entropy, the authors identify a sharp transition governed by training set size, where memorization (vanishing entropy) gives way to generalization (finite entropy).
arXiv:2512. 14338v3 Announce Type: replace Abstract: Many learning problems involve symmetries, and while invariance can be built into neural architectures, it can also emerge implicitly when training on group-structured data.
arXiv:2605. 00366v4 Announce Type: replace-cross Abstract: High-capacity associative memories based on Kernel Logistic Regression (KLR) exhibit strong storage capabilities, but the dynamical and geometric mechanisms underlying their stability remain poorly understood.
arXiv:2606. 10238v1 Announce Type: cross Abstract: Neural population geometry shapes downstream computation.
The paper investigates associative memory in a bipartite Hopfield–Krotov architecture, termed class H, where hidden neurons serve as the retrieval order parameter. Using the replica method, it derives replica‑symmetric phase diagrams and closed‑form capacities for polynomial load, showing that crosstalk statistics are similar for Ising and spherical visible neurons. With a softmax hidden layer, the load becomes exponential, mapping the thermodynamics onto a random‑energy‑model that exhibits paramagnetic, condensed, and frozen phases, and revealing that heating destabilizes retrieval through quantized attention reassignments while Gaussian patterns remain metastable at all loads.
How diffusion models circumvent the curse of dimensionality to learn complex distributions over high dimensional spaces from a finite training set, instead of memorizing it, remains a fundamental mystery. To address this, we introduce analytically tractable Bayesian information restricted diffusion (BIRD) models, in which each pixel observes restricted information about noisy data.
arXiv:2603. 26217v2 Announce Type: replace-cross Abstract: Generalized Hopfield models with higher-order or exponential interaction terms are known to have substantially larger storage capacities than the classical quadratic model.
arXiv:2607. 01311v1 Announce Type: new Abstract: Deep learning has outgrown any single mathematical explanation.
arXiv:2607. 08041v1 Announce Type: new Abstract: How diffusion models circumvent the curse of dimensionality to learn complex distributions over high dimensional spaces from a finite training set, instead of memorizing it, remains a fundamental mystery.