The paper argues that neural networks can exhibit modular organization even when individual neurons are not specialized. It introduces a nested hierarchy of modularity—unit-level, orthogonal, and linear—and shows that many vision, language, and RNN models lack unit-level modularity but still display clear functional modularity at the subspace level. The authors demonstrate that these subspaces can be causally validated through ablation and align with brain data, revealing distinct content/style or syntax/semantics separations.
arXiv:2606. 17889v1 Announce Type: cross Abstract: Compositional learning systems must balance plasticity, the ability to acquire new knowledge, with stability, the preservation of previously learned components, especially when tasks share structure and risk interference.
arXiv:2608.30978v1 Announce Type: new Abstract: Modularity in deep neural networks has been proposed as a means of improving both interpretability and training by promoting disentangled representatio...
arXiv:2608. 12398v1 Announce Type: cross Abstract: We propose IM-LEPP (Integrated Multimodal Latent Energy-based Predictive Processing), a hierarchical, energy-based model of multimodal cognition that extends a previously proposed single-modality model (LEPP) to integrate vision and language.
arXiv:2606. 14975v1 Announce Type: cross Abstract: How the wiring and functional organization of cortex shape recurrent computation remains a central question in both neuroscience and machine learning.
The study trains hierarchical Almost-Linear RNNs on 100 diverse cognitive tasks to investigate how tasks interact when learned jointly. It finds that adding nonlinearity and task-feature dimensionality improves performance, and that a hierarchical model outperforms both individually trained models and a single unstructured joint model. The authors also derive task similarity measures from the trained model that relate to task complexity, solvability, transfer learning, and adaptation.
Understanding the relationship between deep visual representations and the human visual system is a fundamental challenge in computational neuroscience. While modern vision models achieve strong performance in image recognition, their correspondence with the hierarchical organization of the human visual cortex remains an open question.
arXiv:2607. 16295v1 Announce Type: cross Abstract: Mechanistic interpretability has made significant strides in understanding neural network representations, with sparse dictionary learning (SDL) methods, most prominently sparse autoencoders, as a central paradigm.
arXiv:2610.07962v1 Announce Type: cross Abstract: Where generalization capacity--the ability to extract context-invariant relational structures--first emerges remains a central question in AI and neu...
Where generalization capacity--the ability to extract context-invariant relational structures--first emerges remains a central question in AI and neuroscience. The foundation for this capacity lies up...
arXiv:2606. 12886v1 Announce Type: cross Abstract: Interleaved thinking, where a unified multimodal model alternates between textual reasoning and visual generation, has shown promise on spatial and physical tasks.
arXiv:2609.38081v1 Announce Type: new Abstract: On a single task, deep networks can learn many solutions, depending on their optimizer, training data, architecture, and hyperparameters. Many of these...
arXiv:2606. 04772v1 Announce Type: cross Abstract: Understanding the relationship between deep visual representations and the human visual system is a fundamental challenge in computational neuroscience.