The paper introduces the Neurogenesis Network (NGN), a differentiable framework that learns the optimal number of ordered structural components in a neural network during training. By using a learnable boundary to select an active prefix of components, NGN can grow from a compact initialization and later discard unused parts. Experiments across MLPs, CNNs, GNNs, Transformers, state‑space models, LoRA, and adapters show that the learned prefixes perform comparably to fixed‑size models, demonstrating that structural capacity can be optimized directly as a count.
arXiv:2505. 09710v4 Announce Type: replace Abstract: We investigate deep morphological neural networks (DMNNs), studying how changes in algebraic structure affect the expressivity and trainability of deep architectures.
arXiv:2605. 15435v2 Announce Type: replace Abstract: Standard deep-learning pipelines usually choose the network architecture before training and keep it fixed throughout optimization.
arXiv:2608. 14443v1 Announce Type: cross Abstract: Neural Architecture Search (NAS) is naturally formulated as a bilevel optimization problem, where the upper-level optimizes the architecture using validation performance and the lower-level trains network parameters using training loss.
arXiv:2512. 12713v2 Announce Type: replace-cross Abstract: Control policies are often implemented with fixed-capacity multilayer perceptrons trained by backpropagation, which require architecture selection in advance and cannot adapt their capacity during learning.
arXiv:2606. 18676v1 Announce Type: new Abstract: Training-free neural architecture search promises efficient discovery of high-performance networks without costly training.
arXiv:2608. 06993v1 Announce Type: cross Abstract: Large-scale pretrained time-series models achieve strong results through large-scale pretraining and task-agnostic representation learning, but they rely on abundant, diverse data that industrial and scientific domains often lack.
The paper introduces the Linear Reusable Neural Bases Architecture (LRNBA), a framework that represents each network block as a linear combination of shared neural bases to improve parameter efficiency and reduce memory cost. Inspired by recurrent neural network designs, LRNBA enables the construction of wider and deeper networks within the same parameter budget. Experiments show that models using LRNBA converge as fast or faster than classical architectures, achieve lower loss, and maintain stable training dynamics.
arXiv:2606. 01666v1 Announce Type: cross Abstract: The scaling of Large Language Models (LLMs) has driven significant performance gains but created substantial challenges in inference efficiency.
arXiv:2606. 00130v2 Announce Type: replace-cross Abstract: Large deep neural networks are costly to store and deploy because inference must move and evaluate many parameters.
arXiv:2610.00420v1 Announce Type: new Abstract: A weight space network (or metanetwork) takes the weights of another neural network as input and predicts properties of it. Most prior work trains such...
arXiv:2505. 10457v3 Announce Type: replace-cross Abstract: Incremental learning is a machine learning paradigm where a model learns from a sequential stream of tasks.
The paper discusses tensorizing neural networks by reshaping dense weight matrices into higher-order tensors and approximating them with low-rank tensor network decompositions. This approach offers promising model compression and introduces bond indices that create new latent spaces, potentially enhancing interpretability. Despite encouraging empirical results, tensorized neural networks remain underused, and the authors call for more research to address practical scaling and adoption challenges.