The paper investigates how Concept Activation Vectors (CAVs), used for concept‑based explanations in AI, vary due to the randomness in sampling training examples. A theoretical analysis shows that the variance of CAVs decreases roughly as 1/N, where N is the number of sampled examples. Experiments across four modalities confirm this relationship, leading to practical guidelines for efficiently applying CAVs.
arXiv:2602.01718v2 Announce Type: replace Abstract: Predicting generalization from quantities available before target-test evaluation remains a central challenge in deep learning. The systematic benc...
The paper presents a tight generalization bound for deterministic neural networks that relies on the quality of their last hidden layer representations rather than traditional parameter-space metrics. By using a parameter‑free k‑nearest‑neighbor classifier to assess representation quality, the bound decomposes into three measurable components—local class geometry, decoder‑network agreement, and training stability—providing both a non‑vacuous certificate and a diagnostic tool for understanding generalization. Experiments show the bound achieves an 8.0‑percentage‑point gap to test error on CIFAR‑10 with ResNet‑18, yields the first non‑vacuous certificates on CIFAR‑100, and certifies 1.4 % on CIFAR‑10 and 23.1 % on 1,000‑class ImageNet using frozen DINOv2 representations.
arXiv:2606. 06772v1 Announce Type: cross Abstract: Understanding the generalization performance of over-parameterized neural networks has become a central topic in deep learning theory.
arXiv:2608.26083v2 Announce Type: replace-cross Abstract: Deep neural networks often exploit spurious associations, a failure known as shortcut learning. Auditing for shortcuts requires testing many...
ICON Decomposition is a new method for explaining deep neural networks by quantifying how much variance each concept explains in a network layer after accounting for all other concepts and the outcome. Unlike previous concept‑based methods that evaluate concepts in isolation, ICON can distinguish genuine model reliance from spurious correlations. Experiments on synthetic data, skin‑lesion, and brain‑imaging models show that ICON recovers concept importance more accurately, isolates truly relied‑upon concepts, and provides sparse explanations validated through retraining and out‑of‑distribution testing.
Over the past decade, deep neural networks (DNNs) have achieved remarkable success on complex machine-learning tasks, yet the theoretical foundations of their performance remain incomplete. From a statistical viewpoint, a natural question is: can DNNs attain feature-learning and prediction consistency comparable to that of classical models?
arXiv:2502.06970v3 Announce Type: replace Abstract: In contemporary deep learning, a prevalent and effective workflow for solving low-data problems is adapting powerful pre-trained foundation models...
arXiv:2602. 23128v2 Announce Type: replace Abstract: Generalization bounds for deep learning models are typically vacuous, not computable or restricted to specific model classes.
arXiv:2606. 16883v1 Announce Type: cross Abstract: Generalization is a critical property of data-driven models, particularly deep learning models deployed in safety-critical applications.
arXiv:2503. 07325v2 Announce Type: replace Abstract: Understanding and certifying the behavior of modern deep neural networks remains a fundamental challenge in reliable machine learning.
The paper introduces a new family of concentration metrics, called α-RC, based on Rényi divergence to assess how the magnitudes of Jacobian minors in multilayer perceptrons are distributed. Unlike the existing Participation Ratio, these metrics are bounded between 0 and 1 regardless of network size, with 0 meaning uniform spread and 1 indicating dominance by a single minor. For α=2, the α-RC metric reduces to a transformed version of the Participation Ratio, and experiments on single-hidden-layer MLPs demonstrate its practical advantages.