arXiv:2505. 09710v4 Announce Type: replace Abstract: We investigate deep morphological neural networks (DMNNs), studying how changes in algebraic structure affect the expressivity and trainability of deep architectures.
arXiv:2607. 23397v1 Announce Type: new Abstract: Hierarchical neural networks are widely used in artificial intelligence, yet their mathematical properties remain incompletely understood.
The paper introduces the Neurogenesis Network (NGN), a differentiable framework that learns the optimal number of ordered structural components in a neural network during training. By using a learnable boundary to select an active prefix of components, NGN can grow from a compact initialization and later discard unused parts. Experiments across MLPs, CNNs, GNNs, Transformers, state‑space models, LoRA, and adapters show that the learned prefixes perform comparably to fixed‑size models, demonstrating that structural capacity can be optimized directly as a count.
Over the past decade, deep neural networks (DNNs) have achieved remarkable success on complex machine-learning tasks, yet the theoretical foundations of their performance remain incomplete. From a statistical viewpoint, a natural question is: can DNNs attain feature-learning and prediction consistency comparable to that of classical models?
arXiv:2605. 15435v2 Announce Type: replace Abstract: Standard deep-learning pipelines usually choose the network architecture before training and keep it fixed throughout optimization.
arXiv:2607. 03148v1 Announce Type: cross Abstract: Activation functions are considered an essential primitive for neural nonlinearity, i.
arXiv:2607. 09967v1 Announce Type: cross Abstract: Many neural networks operations have a multiplicative nature rather than additive: halving or doubling a norm are analogous relatively but require unequal optimization distances when taking linear steps.
arXiv:2606. 00130v2 Announce Type: replace-cross Abstract: Large deep neural networks are costly to store and deploy because inference must move and evaluate many parameters.
arXiv:2606. 09744v1 Announce Type: new Abstract: We study feed-forward ReLU networks with fixed readout and quadratic loss.
arXiv:2510.05606v2 Announce Type: replace Abstract: Fundamental limits to predictability are central to our understanding of many physical and computational systems. In deep learning, training outcom...
arXiv:2606. 29951v1 Announce Type: new Abstract: Interpretable Mesomorphic Neural Networks (IMNs) offer a promising framework that combines the predictive power of deep neural networks with the interpretability of linear models.
arXiv:2403.04545v4 Announce Type: replace Abstract: Scaling factors in residual branches have emerged as a prevalent method for boosting neural network performance, especially in normalization-free a...
arXiv:2606. 00888v1 Announce Type: cross Abstract: Dynamic Sparse Training (DST) offers a promising paradigm for improving the training and inference efficiency of deep neural networks; however, we find that in large language model training, DST can suffer from optimization instability, manifested as loss spikes after topology updates.