The paper investigates how single‑hidden‑layer MLPs can fit training data yet fail to recover the underlying rule, focusing on higher‑order interactions and nuisance inputs. Using synthetic parity tasks, the authors benchmark different optimizers (SGD, Adam, Muon) and show that while all achieve perfect accuracy on second‑order interactions, performance drops sharply for higher orders, with Muon outperforming the others at fourth order. Experiments also reveal that freezing or removing nuisance‑related weights dramatically alters training outcomes, highlighting the role of nuisance learning in shaping the rules a shallow network can represent.
By Gongyue Zhang, Honghai Liu
arXiv:2607. 11914v1 Announce Type: cross Abstract: A central goal of current Spiking Neural Network (SNN) research is to improve their accuracy toward becoming low-power alternatives to Artificial Neural Networks (ANNs).
By Jiahong Zhang, Sijun Shen, Man Yao, Han Xu, Mingqiang Huang, Yonghong Tian, Bo Xu, Guoqi Li
arXiv:2609.26167v1 Announce Type: new
Abstract: Activation-energy pruning -- removing weights whose product of magnitude and cumulative pre-synaptic spike count falls below a threshold -- was establi...
By Joseph Bingham
arXiv:2609.23561v1 Announce Type: new
Abstract: Deploying efficient neural networks is essential in resource-constrained environments, yet compact models often sacrifice interpretability - a critical...
By Aleks Czufarow, Ihor Babin
arXiv:2602.01718v2 Announce Type: replace
Abstract: Predicting generalization from quantities available before target-test evaluation remains a central challenge in deep learning. The systematic benc...
By Sora Nakai, Youssef Fadhloun, Kacem Mathlouthi, Kotaro Yoshida, Ganesh Talluri, Ioannis Mitliagkas, Hiroki Naganuma
arXiv:2606. 05740v1 Announce Type: new Abstract: Deep neural networks trained under severe class imbalance often exhibit degraded performance, typically attributed to statistical bias.
By Arush Singhal, Umang Soni
arXiv:2606. 30676v1 Announce Type: cross Abstract: Deploying spiking neural networks (SNNs) on neuromorphic hardware demands aggressive synaptic pruning while preserving temporal computation integrity.
By Muhammad Hamza
arXiv:2606. 11123v1 Announce Type: new Abstract: Backpropagation (BP) is widely viewed as biologically implausible, in part because it requires feedback weights to be the transpose of forward weights for error propagation.
By Gauthier Boeshertz, Razvan Pascanu, Claudia Clopath
arXiv:2608. 14922v1 Announce Type: cross Abstract: Mechanistic interpretability has recently expanded to Vision Transformers (ViTs), with Sparse Autoencoders (SAEs) increasingly used as post-hoc tools to decompose internal representations into sparse and more interpretable features.
By Philip H. Lee, Parth Padalkar
arXiv:2505. 03303v3 Announce Type: replace-cross Abstract: Lightweight convolutional neural networks are often compared using results obtained with different training recipes, input settings, and pretrained checkpoints.
By Tasnim Shahriar
arXiv:2608. 13513v1 Announce Type: cross Abstract: Tabular-to-image methods have emerged as novel approaches to leverage the high predictive performance of convolutional neural networks and vision transformers.
By David Chushig-Muzo, Mar\'ia \'Angeles Rodr\'iguez de Cara, Eva Milara, Francisco J. Lara-Abelenda, Luis Zhinin-Vera, Diego H. Peluffo-Ord\'o\~nez
arXiv:2606. 27855v1 Announce Type: cross Abstract: Deep learning models for surface electromyography (sEMG) can benefit substantially from subject-specific (re-)calibration, since no sufficiently large and diverse datasets are available to train fully generic decoders.
By Stephan J. Lehmler, Tobias Glasmachers, Ioannis Iossifidis