← Back to all news
arXiv Machine Learning By Zekun Wang, Karthik Singaravadivelan, Christopher J. MacLellan

Diffusion Models and Concept Formation

Read the original on arXiv Machine Learning →

The paper argues that diffusion models, originally developed for image synthesis, implicitly perform concept formation similar to the Cobweb cognitive model. Both models build hierarchical density structures using Gaussian prototypes, treat categorization as score‑following to reduce uncertainty, and exhibit a basic level of abstraction. The authors demonstrate this correspondence by extracting a diffusion hierarchy from MNIST and Fashion‑MNIST data and comparing its basic level to that of Cobweb, suggesting diffusion models can serve as a continuous, scalable instantiation of concept formation.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

Hugging Face Trending Papers
Jul 9

An exact information theory of generalization phase transitions in Bayesian diffusion models

How diffusion models circumvent the curse of dimensionality to learn complex distributions over high dimensional spaces from a finite training set, instead of memorizing it, remains a fundamental mystery. To address this, we introduce analytically tractable Bayesian information restricted diffusion (BIRD) models, in which each pixel observes restricted information about noisy data.

diffusion
More like this →
arXiv Machine Learning
Sep 14

Hierarchical Prototype Emergence in Modern Hopfield Models

The paper studies how hierarchical correlations in data can be learned by a dense Hopfield network with polynomial activation. It analytically derives conditions for each level of a hierarchical memory model to be locally stable, meaning they correspond to local energy minima. Using prototype reconstruction as a minimal generalization test, the authors show that only a quasi‑polynomial amount of information is needed to generalize beyond specific memories or groups, and they observe a similar phase diagram for Fashion‑MNIST data.

By Aditya Cowsik, Adithya Sriram
diffusion
More like this →