NeuronSpark is a self‑attention‑free language model that uses input‑dependent spiking membrane dynamics, allowing each neuron’s decay rate, gain, and firing threshold to adapt to the current token. This selective spiking approach enables billion‑parameter pretraining from random initialization without relying on backbone transfer, conversion, or distillation, achieving competitive zero‑shot performance and a fast–slow hierarchy. On a 1.1B‑parameter, 2.1B‑token training run, NeuronSpark attains a 40.0 eight‑task zero‑shot average, surpasses comparable‑budget Pythia/Mamba checkpoints, and demonstrates significant energy and cycle reductions on FPGA implementations.
By Zhengzheng Tang
The paper presents a spiking neural network (SNN) approach that uses time-to-first-spike (TTFS) coding to limit each neuron to at most one spike per time window, enabling energy-efficient large language models (LLMs). A reference-based strategy is introduced to encode the four core LLM components—embedding layers, layer normalization, attention-related operations, and dropout—allowing the construction of a fully TTFS-based SNN architecture trained end-to-end. Experiments on BERT and GPT-2 show performance comparable to artificial neural network (ANN) counterparts on natural language understanding and common-sense reasoning, while achieving a 1.5‑billion‑parameter spiking LLM and providing an estimate of spike-related energy consumption.
By Zhuoya Zhao, Parsa Omidi, Aref Jafari, Richard Naud
The paper introduces the Spiking Ternary GRU (stGRU), a bio‑inspired architecture that couples ternary‑weight gated recurrent units with resonate‑and‑fire neurons to enable efficient, matrix‑MAC‑free spiking neural networks for speech recognition. Trained end‑to‑end on LibriSpeech, the stGRU achieves single‑digit word error rates (6.44% on test‑clean) without an external language model, and also attains competitive character error rates on Aishell‑1. The model develops a depth‑graded hierarchy where shallow layers filter acoustic signals and deeper layers integrate semantic context, mirroring hierarchical processing in human cortex.
By Pavlos Poulos, Katelyn Sadorf, Themos Stafylakis, Costas A. Anastassiou
arXiv:2512. 01906v3 Announce Type: replace Abstract: Spiking neural networks (SNNs) are biologically inspired, event-driven models suited for temporal data processing and energy-efficient neuromorphic computing.
By Sanja Karilanova, Subhrakanti Dey, Ay\c{c}a \"Oz\c{c}elikkale
Spiking neural networks (SNNs) are promoted as an energy-efficient substrate because sparse, event-driven activity replaces dense multiply-accumulates with cheap accumulates. We argue the energy dividend of sparsity is not a property of SNNs but of the task.
arXiv:2607. 26648v1 Announce Type: cross Abstract: Spiking neural networks (SNNs) are promoted as an energy-efficient substrate because sparse, event-driven activity replaces dense multiply-accumulates with cheap accumulates.
By Zeyu Wang
arXiv:2602.01124v4 Announce Type: replace
Abstract: Dynamic graph representation learning requires capturing both structural relations and temporal evolution, yet existing approaches face a core trad...
By Md Abrar Jahin, Taufikur Rahman Fuad, Jay Pujara, Craig Knoblock
arXiv:2607. 02283v1 Announce Type: cross Abstract: In-context learning (ICL) operates via implicit gradient descent embedded in the forward pass of modern AI architectures -- Transformers, Mamba, state-space models, and MLPs.
By Juwei Shen, Yujie Wu, Changwen Chen
arXiv:2605.12049v2 Announce Type: replace-cross
Abstract: Cortical neurons are complex, multi-timescale processors wired into recurrent circuits, shaped by long evolutionary pressure under stringent...
By Aaron Spieler, Georg Martius, Anna Levina
The paper proposes a biologically inspired micro‑sleep technique called napping for recurrent spiking neural networks, combining proportional weight scaling with continuous stochastic membrane activity. Experiments on an unsupervised SNN trained with trace‑based STDP on Gabor‑preprocessed MNIST show that well‑tuned napping can match the classification accuracy of conventional weight normalization while offering different clustering characteristics. The study suggests that napping may be preferable when representational structure is more important than raw classification speed, despite its higher simulation cost.
By Andreas Massey, Stefano Nichele, Aliaksandr Hubin
The paper introduces the Dormant Spiking Neuron (DSN), a state-driven mechanism that allows neurons in spiking neural networks to enter a dormant state when their membrane potential falls below a negative threshold. This dynamic gating prunes redundant computations at the instance level and is trained end‑to‑end via backpropagation through time with surrogate gradients. Experiments on static and neuromorphic datasets, including ImageNet, show that DSN reduces computational energy by 40% to over 70% and improves baseline SNN accuracy by 1.34%, outperforming state‑of‑the‑art static pruning methods.
By Ruichen Ma, Yicai Chen, Pujun Zhou, Yue Zuo, Chongyang Li, Shuang Liu, Shaogang Hu, Guanchao Qiao
arXiv:2610.09710v1 Announce Type: new
Abstract: ANN-to-SNN conversion offers a practical route toward energy-efficient spiking Vision-Language-Action (VLA) models by bypassing the substantial cost of...
By Jingya Wang, Dehao Zhang, Shuai Wang, Malu Zhang, Yang Yang, Haizhou Li