arXiv Machine Learning

Quadratic integrate-and-fire neurons exhibit less fragmented loss landscapes and outperform leaky integrate-and-fire neurons in spike-based gradient descent

arXiv:2606. 03935v1 Announce Type: cross Abstract: The ability to train spiking neural networks is essential for modeling biological neural networks as well as for neuromorphic computing.

arXiv AI
Sep 16

Equivalence of approximation by networks of single- and multi-spike neurons

The paper demonstrates that for a broad class of spiking neuron models, including the leaky integrate‑and‑fire with subtractive reset, any approximation bound proven for multi‑spike networks can be translated to an equivalent single‑spike network with only a linear change in neuron count, and vice versa. This establishes that single‑spike and multi‑spike neural networks possess identical approximation capabilities for general machine learning tasks. Consequently, existing approximation results for single‑spike networks automatically extend to the multi‑spike case.

By Dominik Dold, Philipp Christian Petersen
arXiv Machine Learning
Jun 11

A2SG:Adaptive and Asymmetric Surrogate Gradients for Training Deep Spiking Neural Networks

arXiv:2606. 11236v1 Announce Type: cross Abstract: Training deep spiking neural networks (SNNs) remains challenging due to sharp loss landscapes and temporal inconsistency caused by surrogate gradients.

By Yechan Kang, Yongjin Kweon, Mingyeong Seo, Sohee Park, Yeonguk Jeon, Jongkil Park, Hyun Jae Jang, Jaewook Kim, YeonJoo Jeong, Suyoun Lee, Seongsik Park
arXiv Machine Learning
Sep 15

Exploring napping paradigm for Recurrent Spiking Neural Networks

The paper proposes a biologically inspired micro‑sleep technique called napping for recurrent spiking neural networks, combining proportional weight scaling with continuous stochastic membrane activity. Experiments on an unsupervised SNN trained with trace‑based STDP on Gabor‑preprocessed MNIST show that well‑tuned napping can match the classification accuracy of conventional weight normalization while offering different clustering characteristics. The study suggests that napping may be preferable when representational structure is more important than raw classification speed, despite its higher simulation cost.

By Andreas Massey, Stefano Nichele, Aliaksandr Hubin
arXiv Machine Learning
5d ago

AYLA: Architecting a loss landscape in shallow neural networks to accelerate feature recovery

AYLA is a loss reparameterization framework that applies a sigmoid‑controlled power‑law transformation to the empirical loss, dynamically adjusting gradient magnitudes without changing stationary points or optimal solutions. By reshaping optimization trajectories, AYLA accelerates descent in flat or saddle‑dominated regions and stabilizes late‑stage training, leading to improved feature recovery in two‑layer tanh networks on synthetic Gaussian data. Experiments show enhanced weight alignment, neuron similarity, activation correlation, and richer internal representations, while mitigating rank collapse and promoting a transition from lazy to active feature‑learning regimes.

By Behnam Gheshlaghi, Shahin Atakishiyev