The paper investigates whether neural networks exhibit conceptual separation, meaning that examples of the same concept cluster together and related concepts are closer in representation space. Using geometric and distributional analyses, the authors find that Convolutional Neural Networks (CNNs) produce coherent, semantically ordered representations for familiar ImageNet concepts, but this coherence weakens for unseen concepts and under domain shift. Large Language Models (LLMs) keep distinct domains well separated, bring related subdomains closer, yet lose distinction between ambiguous topics at both mean and covariance levels.
arXiv:2606. 29416v1 Announce Type: cross Abstract: Can a vision model truly see an object, or does it only fit surface-level visual cues?
arXiv:2512. 11000v2 Announce Type: replace-cross Abstract: Representations pervade our daily experience, from letters representing sounds to bit strings encoding digital files.
arXiv:2609.14151v1 Announce Type: cross Abstract: Steering language models using a set of contrastive representations has been a canonical and computationally efficient method for controlling model b...
arXiv:2512. 15134v2 Announce Type: replace-cross Abstract: A goal of interpretability is to recover disentangled representations of latent concepts (features) from the activations of neural networks.
arXiv:2607. 04525v1 Announce Type: cross Abstract: How concepts are represented in neural networks is a fundamental question in machine learning.
How concepts are represented in neural networks is a fundamental question in machine learning. The dominant view treats concept representations as stationary geometric objects.
arXiv:2607. 10578v1 Announce Type: new Abstract: Existing hypotheses represent a concept in an LLM as a single point, a linear direction, or a Gaussian cluster, yet it remains unclear how and why such structures emerge.
arXiv:2510. 15770v4 Announce Type: replace-cross Abstract: Although deep neural networks achieve strong predictive performance, their internal reasoning often remains difficult to inspect and control.
arXiv:2609.13198v1 Announce Type: new Abstract: One of the pivotal recent challenges in neural network interpretability is polysemanticity, where a single neuron is activated by multiple, often unrel...
arXiv:2602. 24264v2 Announce Type: replace-cross Abstract: Compositional generalization, the ability to recognize familiar parts in novel contexts, is a defining property of intelligent systems.
arXiv:2606. 30498v1 Announce Type: cross Abstract: Human decision-making interprets the world through high-level concepts, such as recognizing a bird by its belly color.
ICON Decomposition is a new method for explaining deep neural networks by quantifying how much variance each concept explains in a network layer after accounting for all other concepts and the outcome. Unlike previous concept‑based methods that evaluate concepts in isolation, ICON can distinguish genuine model reliance from spurious correlations. Experiments on synthetic data, skin‑lesion, and brain‑imaging models show that ICON recovers concept importance more accurately, isolates truly relied‑upon concepts, and provides sparse explanations validated through retraining and out‑of‑distribution testing.