arXiv:2608. 14443v1 Announce Type: cross Abstract: Neural Architecture Search (NAS) is naturally formulated as a bilevel optimization problem, where the upper-level optimizes the architecture using validation performance and the lower-level trains network parameters using training loss.
arXiv:2608. 14472v1 Announce Type: cross Abstract: Neural Architecture Search (NAS) aims to automate neural network architecture design, reducing reliance on human expertise.
arXiv:2606. 29582v1 Announce Type: cross Abstract: Bilevel optimization has become an influential and widely adopted framework for addressing hierarchical optimization problems in machine learning, providing an effective approach to modeling the interaction between two levels of optimization, with applications such as hyperparameter tuning, meta-learning, adversarial training, and data poisoning.
arXiv:2603. 15106v2 Announce Type: replace Abstract: Enabling efficient deep neural network (DNN) inference on edge devices with different hardware constraints is a challenging task that typically requires DNN architectures to be specialized for each device separately.
ONNX-Net introduces a universal representation for neural architectures using natural language descriptions, enabling instant performance prediction across diverse search spaces. The authors present ONNX-Bench, a benchmark of over 600k architecture–accuracy pairs compiled from open‑source NAS‑bench networks in ONNX format. Experiments demonstrate strong zero‑shot predictive performance with minimal pretraining, overcoming the limitations of cell‑based, graph‑encoded approaches.
arXiv:2607. 15745v1 Announce Type: new Abstract: Common practice when training Convolutional Neural Networks (CNNs) is to use randomly shuffled mini-batches.
The paper compares genetic algorithm (GA) and gradient descent (GD) training for a distance‑encoding biomorphic‑informational neural network (DEBI‑NN) designed for low‑data medical datasets. A spatial backpropagation scheme was implemented for GD, and both optimizers were evaluated on synthetic, radiomic, and fetal cardiotocography datasets. Across all experiments, GA consistently outperformed GD, achieving higher classification accuracy and more stable decision boundaries, while GD struggled with the interdependent spatial parameters of DEBI‑NN.
arXiv:2606. 00130v2 Announce Type: replace-cross Abstract: Large deep neural networks are costly to store and deploy because inference must move and evaluate many parameters.
arXiv:2605. 15435v2 Announce Type: replace Abstract: Standard deep-learning pipelines usually choose the network architecture before training and keep it fixed throughout optimization.
arXiv:2607. 09399v1 Announce Type: cross Abstract: We introduce a novel method for both partial and full optimization of the connections in deep differentiable logic gate networks (LGNs) and lookup table networks (LUTNs).
arXiv:2602. 06737v2 Announce Type: replace Abstract: We present a generalized framework for the range verification of neural networks featuring non-linear activation functions.
The paper introduces the Neurogenesis Network (NGN), a differentiable framework that learns the optimal number of ordered structural components in a neural network during training. By using a learnable boundary to select an active prefix of components, NGN can grow from a compact initialization and later discard unused parts. Experiments across MLPs, CNNs, GNNs, Transformers, state‑space models, LoRA, and adapters show that the learned prefixes perform comparably to fixed‑size models, demonstrating that structural capacity can be optimized directly as a count.
arXiv:2607. 06772v1 Announce Type: new Abstract: Learned optimization aims to improve upon hand-designed optimizers (e.