arXiv:2405. 02369v2 Announce Type: replace-cross Abstract: In the past decade, many successful networks are on novel architectures, which almost exclusively use the same type of neurons.
arXiv:2607. 19973v1 Announce Type: new Abstract: AI researchers describe state-of-the-art models as one thing repeated at scale: the Transformer, wired identically for text, pixels, or speech.
arXiv:2606. 06624v1 Announce Type: new Abstract: In the current era of deep learning and especially generative models, there is significant investment in training very large generative models.
arXiv:2501.18018v2 Announce Type: replace-cross Abstract: The neurons of artificial neural networks were originally invented when much less was known about biological neurons than is known today. Our...
arXiv:2607. 07743v1 Announce Type: cross Abstract: Self-organization is an emergent property of life, driven by the collective behavior of individual components acting on local information.
arXiv:2608. 04358v1 Announce Type: new Abstract: Continual learning (CL) requires models to learn tasks sequentially, yet deep neural networks often suffer from plasticity loss and poor knowledge transfer, which can impede their long-term adaptability.
FlowGenesis is a framework that lets neural networks self‑organize their architecture during training by adding or removing neurons based on a local statistic called Representation Flux. Neurons with high Flux are split to preserve function, while low‑Flux, low‑activity neurons are pruned. Experiments on image and text classification show that networks starting from different sizes converge to similar neuron counts and performance, and the method also improves continual learning by reducing parameters for new tasks without sacrificing accuracy.
arXiv:2606. 23587v2 Announce Type: replace Abstract: Previous work has found a gap between the scale of neural networks that reliably learn Conway's Game of Life, and minimal networks capable of representing the classic cellular automaton with hard-coded parameter values.
arXiv:2510. 22450v3 Announce Type: replace-cross Abstract: The choice of activation function plays a critical role in neural networks, yet most architectures still rely on fixed, uniform activation functions across all neurons.
The paper investigates how multilayer perceptrons (MLPs) learn features in regression tasks with clustered data. It finds that instead of forming a single global low‑dimensional representation, MLPs develop monosemantic specialized neurons—each neuron aligns strongly with a specific predictive feature relevant to a particular region of the input space. This specialization results in a collection of local low‑dimensional representations, giving MLPs a provable data‑efficiency advantage over methods that rely on a global representation.
arXiv:2606. 31819v1 Announce Type: new Abstract: This work introduces a new computational theory of mind grounded in set theory and hyperdimensional computing.
The paper investigates the relationship between representation and function in neural networks, using analytical two‑layer linear models and simulations of nonlinear networks. It demonstrates that functional similarity and representational similarity can be dissociated: networks may share representations without sharing functions, and vice versa. The study also finds that robustness to input noise or generalization error does not constrain representations, whereas robustness to parameter noise forces networks to adopt task‑specific representations, indicating that representational alignment reflects computational advantages beyond mere functional alignment.