OrthoRF introduces orthogonality into Rotating Features (RF) by applying an inner-product loss and architectural changes, improving phase alignment and grouping reliability. The method outperforms or matches existing unsupervised object discovery models, especially in challenging scenarios such as overlapping objects, noise, and out-of-distribution tests. Additionally, OrthoRF can recover occluded object parts without requiring post-hoc clustering, producing more interpretable representations.
By Despoina Touska, Bastiaan Onne Fagginger Auer, Alexandru Onose, Tejaswi Kasarla, Luis Armando P\'erez Rey, Maximilian Lipp, Lyubov Amitonova, Martin R. Oswald, Pascal Cerfontaine
The paper investigates how multilayer perceptrons (MLPs) learn features in regression tasks with clustered data. It finds that instead of forming a single global low‑dimensional representation, MLPs develop monosemantic specialized neurons—each neuron aligns strongly with a specific predictive feature relevant to a particular region of the input space. This specialization results in a collection of local low‑dimensional representations, giving MLPs a provable data‑efficiency advantage over methods that rely on a global representation.
By Amirhesam Abedsoltan, Enric Boix-Adsera, Fivos Kalogiannis, Mikhail Belkin
arXiv:2608.24007v1 Announce Type: new
Abstract: Understanding how neural networks learn and organize features is central to understanding their behavior. Much existing theory of feature learning has...
By Amirhesam Abedsoltan, Enric Boix-Adsera, Fivos Kalogiannis, Mikhail Belkin
The paper introduces TPR-Attention, an attention mechanism that operates over tensor‑product representations to embed structured inductive bias into deep learning models. Experiments on compositional tasks demonstrate that TPR‑Attention outperforms existing architectural components in achieving combinatorial generalization. The results suggest that incorporating explicit compositional structure into neural attention can improve systematic generalization.
By Melisa Civeleko\u{g}lu, Isabeau Pr\'emont-Schwarz
arXiv:2605.05556v2 Announce Type: replace
Abstract: Artificial neural networks trained on visual tasks develop internal representations resembling those of the primate visual system, a discovery that...
By Yash Mehta, Michael F. Bonner
arXiv:2505.09716v4 Announce Type: replace
Abstract: Out-of-distribution (OOD) generalisation through composition requires a system to discover invariant properties from input-output associations and...
By George Dimitriadis, Spyridon Samothrakis
arXiv:2505.09716v3 Announce Type: replace-cross
Abstract: Out-of-distribution (OOD) generalisation is considered a hallmark of human and animal intelligence. To achieve OOD through composition, a sys...
By George Dimitriadis, Spyridon Samothrakis
Systematic generalization remains a significant challenge in deep learning. In particular, combinatorial generalization - generalizing to new configurations of known factors of variation - is effortle...
arXiv:2606. 28399v1 Announce Type: cross Abstract: The structure of human visual representations underpins our capacity for adaptive behaviour.
By Can Demircan, Marcel Binz, Alireza Modirshanechi, Eric Schulz
arXiv:2607. 00620v1 Announce Type: cross Abstract: Generalized Category Discovery (GCD) aims to recognize known classes while autonomously discovering novel ones in open-world settings.
By Boyang Dai, Chaoqi Chen, Yizhou Yu
arXiv:2605. 07821v2 Announce Type: replace-cross Abstract: Out-of-distribution (OOD) detection is crucial for ensuring the reliability of deep learning models.
By Boyang Dai, Chaoqi Chen, Yizhou Yu
arXiv:2603. 13994v2 Announce Type: replace-cross Abstract: Vision foundation models trained with self-supervised objectives achieve strong performance across diverse tasks and exhibit emergent object segmentation properties.
By Hossein Adeli, Seoyoung Ahn, Andrew Luo, Mengmi Zhang, Nikolaus Kriegeskorte, Gregory Zelinsky