← Back to all news
arXiv AI By Himanshu Udupi, Calvin Ang, Sungyoon Kim, Xiaocong Yang, ChengXiang Zhai

Convex Optimization of Spiking Neural Networks with Global and Approximation Guarantees

Read the original on arXiv AI →

The paper presents a convex formulation for training spiking neural networks (SNNs), showing that the weight‑decay‑regularized training problem for a class of threshold RNNs—including leaky integrate‑and‑fire SNNs—has zero duality gap and matches a convex program’s optimal value. To handle the exponential size of the resulting convex programs, the authors introduce a randomized surrogate with a high‑probability approximation guarantee. Empirical results demonstrate that the convex variant outperforms surrogate‑gradient training on addition and first‑last‑XOR tasks, matches performance on sequential MNIST, and falls slightly behind a reservoir baseline on base‑3 and base‑5 addition.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Machine Learning
Jun 11

A2SG:Adaptive and Asymmetric Surrogate Gradients for Training Deep Spiking Neural Networks

arXiv:2606. 11236v1 Announce Type: cross Abstract: Training deep spiking neural networks (SNNs) remains challenging due to sharp loss landscapes and temporal inconsistency caused by surrogate gradients.

By Yechan Kang, Yongjin Kweon, Mingyeong Seo, Sohee Park, Yeonguk Jeon, Jongkil Park, Hyun Jae Jang, Jaewook Kim, YeonJoo Jeong, Suyoun Lee, Seongsik Park
llmscomputer-vision
More like this →
arXiv AI
Aug 17

SAGE: Surrogate-gradient Adaptation via Attention-Guided Entropy for Spiking Transformers

arXiv:2608. 13702v1 Announce Type: cross Abstract: Spiking neural networks (SNNs) offer an energy-efficient alternative to conventional deep neural networks by exploiting sparse event-driven computation, but their training remains challenging because the non-differentiable spike function requires surrogate gradients whose fixed shape may be suboptimal across layers and training stages.

By Kiran Nair, Rodrigue Rizk, KC Santosh
llms
More like this →
arXiv Machine Learning
Sep 25

On the second-order optimization for spiking neural networks

The paper introduces SpiKFAX, a second‑order optimization technique for Spiking Neural Networks (SNNs) that uses a Kronecker‑factored approximation of the Fisher information matrix tailored to the sparse, discrete, and temporally recurrent dynamics of SNNs. By addressing the sharp loss landscape that hampers training with conventional optimizers, SpiKFAX improves test accuracy and training stability across five architectures and seven datasets. The method offers a computationally tractable alternative to existing curvature‑based approaches for SNNs.

By Ngoc Phu Doan, Ihsen Alouani
More like this →