arXiv:2606. 21593v2 Announce Type: replace Abstract: Deep neural networks transform input data into latent representations that support a wide range of downstream tasks.
By Linara Adilova, Henning Petzka, Asja Fischer, Bernhard C. Geiger
The paper proposes a structured version of the Information Bottleneck (IB) that separates label-relevant structure from within-condition variation using a dual-bottleneck formulation. It introduces a conditional KL term that targets within-condition information, allowing explicit control over nuisance-like variation in learned representations. Experiments demonstrate improved performance in low-data classification and consistent gains on dense prediction tasks.
By Jingyao Zhang, Yuxuan Li, Lu Han, Ali Anaissi, Nguyen H. Tran
arXiv:2603. 27631v2 Announce Type: replace Abstract: Self-supervised pre-training, where large corpora of unlabeled data are used to learn representations for downstream fine-tuning, has become a cornerstone of modern machine learning.
By Mohammad Tinati, Stephen Tu
arXiv:2607. 03145v1 Announce Type: cross Abstract: The informativeness of a training set is as consequential as its size, yet most sampling strategies remain agnostic to the intrinsic geometry of the data distribution.
By Alexandre L. M. Levada
arXiv:2610.00751v1 Announce Type: cross
Abstract: Recent theoretical work identified fundamental properties of representation geometry that shape inference ability of deep neural networks. These incl...
By Sakin Kirti, Joel Zylberberg
The paper introduces a pointwise generalization theory for fully connected deep neural networks, using a pointwise Riemannian Dimension derived from eigenvalues of learned feature representations across layers. This framework provides hypothesis-dependent, representation-aware generalization bounds that are significantly tighter than traditional size- or norm-based approaches, both theoretically and experimentally. The authors analytically identify structural properties that explain deep networks’ tractability and empirically show that the pointwise Riemannian Dimension captures feature compression, over‑parameterization effects, and optimizer bias.
By Shaojie Li, Yunbei Xu