SpikeMoE introduces a spike-based k‑WTA router that uses lateral inhibition and refractory periods to select the top‑K experts based on discrete spike counts, inspired by hippocampal CA1 competition. The framework combines spiking neural network dynamics with mixture‑of‑experts conditional computation and adds a two‑stage missing‑modality module for robust multimodal processing. Experiments on vision, language, and multimodal tasks show that SpikeMoE matches or surpasses ANN baselines while offering energy‑efficient performance.
By Xiaoli Liu, Yujie Liang, Jialin Li, Malu Zhang
arXiv:2606. 14975v1 Announce Type: cross Abstract: How the wiring and functional organization of cortex shape recurrent computation remains a central question in both neuroscience and machine learning.
By Mo Shakiba, Rana Rokni, Mohammad Mohammadi, Nima Dehghani
arXiv:2607. 02283v1 Announce Type: cross Abstract: In-context learning (ICL) operates via implicit gradient descent embedded in the forward pass of modern AI architectures -- Transformers, Mamba, state-space models, and MLPs.
By Juwei Shen, Yujie Wu, Changwen Chen
arXiv:2504.14015v2 Announce Type: replace-cross
Abstract: We introduce "causal pieces", a novel concept for analysing spiking neural networks (SNNs), inspired by "linear pieces" used to study express...
By Dominik Dold, Philipp Christian Petersen
arXiv:2609.21583v1 Announce Type: cross
Abstract: Feedforward Spiking Neural Networks (SNNs) typically propagate every generated spike indiscriminately, disregarding whether the information is redund...
By Aidin Attar, Michele Rossi
arXiv:2608. 19238v1 Announce Type: cross Abstract: Spiking Transformers model token interactions primarily through spiking self-attention (SSA).
By Dongcheng Zhao, Sicheng Shen, Zhenyu Yang, Zhiyuan Li, Jinyan Yu, Yongjian Wang, Tiechui Yao, Wenli Zhang, Tielin Zhang