The paper presents a spiking neural network (SNN) approach that uses time-to-first-spike (TTFS) coding to limit each neuron to at most one spike per time window, enabling energy-efficient large language models (LLMs). A reference-based strategy is introduced to encode the four core LLM components—embedding layers, layer normalization, attention-related operations, and dropout—allowing the construction of a fully TTFS-based SNN architecture trained end-to-end. Experiments on BERT and GPT-2 show performance comparable to artificial neural network (ANN) counterparts on natural language understanding and common-sense reasoning, while achieving a 1.5‑billion‑parameter spiking LLM and providing an estimate of spike-related energy consumption.
By Zhuoya Zhao, Parsa Omidi, Aref Jafari, Richard Naud
arXiv:2512. 01906v3 Announce Type: replace Abstract: Spiking neural networks (SNNs) are biologically inspired, event-driven models suited for temporal data processing and energy-efficient neuromorphic computing.
By Sanja Karilanova, Subhrakanti Dey, Ay\c{c}a \"Oz\c{c}elikkale
Spiking neural networks (SNNs) are promoted as an energy-efficient substrate because sparse, event-driven activity replaces dense multiply-accumulates with cheap accumulates. We argue the energy dividend of sparsity is not a property of SNNs but of the task.
arXiv:2607. 26648v1 Announce Type: cross Abstract: Spiking neural networks (SNNs) are promoted as an energy-efficient substrate because sparse, event-driven activity replaces dense multiply-accumulates with cheap accumulates.
By Zeyu Wang
arXiv:2607. 02283v1 Announce Type: cross Abstract: In-context learning (ICL) operates via implicit gradient descent embedded in the forward pass of modern AI architectures -- Transformers, Mamba, state-space models, and MLPs.
By Juwei Shen, Yujie Wu, Changwen Chen
The paper proposes a biologically inspired micro‑sleep technique called napping for recurrent spiking neural networks, combining proportional weight scaling with continuous stochastic membrane activity. Experiments on an unsupervised SNN trained with trace‑based STDP on Gabor‑preprocessed MNIST show that well‑tuned napping can match the classification accuracy of conventional weight normalization while offering different clustering characteristics. The study suggests that napping may be preferable when representational structure is more important than raw classification speed, despite its higher simulation cost.
By Andreas Massey, Stefano Nichele, Aliaksandr Hubin
Spiking Neural Networks (SNNs) are well-regarded for their biological plausibility and energy efficiency in processing sequential data. However, dominant SNN architectures typically rely on first-order Ordinary Differential Equations (ODEs) to govern neuronal state transitions.
arXiv:2606. 03935v1 Announce Type: cross Abstract: The ability to train spiking neural networks is essential for modeling biological neural networks as well as for neuromorphic computing.
By Carlo Wenig, Raoul-Martin Memmesheimer, Christian Klos
arXiv:2605.21333v3 Announce Type: replace-cross
Abstract: Natively trained spiking language models must preserve information across time while operating through sparse binary activations, a combinati...
By Ting Liu
arXiv:2607. 14672v1 Announce Type: new Abstract: Continuous-time spiking neural networks (SNNs) provide an event-driven framework for temporal computation, computational neuroscience, and neuromorphic hardware.
By Yusuke Sakemi, Tomoya Takeuchi, Takeo Hosomi, Kazuyuki Aihara
arXiv:2608. 08479v1 Announce Type: cross Abstract: Spiking neural networks (SNNs) offer a promising pathway to energy-efficient AI and brain-inspired computing.
By Prasanna Date, Kevin Zhu, Shruti Kulkarni, Ashish Gautam, Chathika Gunaratne, Robert Patton, Tyler Nitzsche, Ian Mulet, Zachary Johnson-Scott, Addison Helms, Duncan Rowden, Simon Weston, Maryam Parsa, Catherine Schuman, Thomas Potok
arXiv:2606. 12287v1 Announce Type: cross Abstract: The Transformer architecture is widely regarded as the most powerful tool for natural language processing, but due to a high number of complex operations, it inherently faces the issue of high energy consumption.
By Claas Beger, Florian Walter, Alois Knoll