Spiking neural networks (SNNs) are promoted as an energy-efficient substrate because sparse, event-driven activity replaces dense multiply-accumulates with cheap accumulates. We argue the energy dividend of sparsity is not a property of SNNs but of the task.
arXiv:2607. 26648v1 Announce Type: cross Abstract: Spiking neural networks (SNNs) are promoted as an energy-efficient substrate because sparse, event-driven activity replaces dense multiply-accumulates with cheap accumulates.
arXiv:2609.13899v1 Announce Type: cross Abstract: The BabyLM challenge measures how much language a model can learn from developmentally-plausible, child-scale data rather than internet-scale corpora...
NeuronSpark is a self‑attention‑free language model that uses input‑dependent spiking membrane dynamics, allowing each neuron’s decay rate, gain, and firing threshold to adapt to the current token. This selective spiking approach enables billion‑parameter pretraining from random initialization without relying on backbone transfer, conversion, or distillation, achieving competitive zero‑shot performance and a fast–slow hierarchy. On a 1.1B‑parameter, 2.1B‑token training run, NeuronSpark attains a 40.0 eight‑task zero‑shot average, surpasses comparable‑budget Pythia/Mamba checkpoints, and demonstrates significant energy and cycle reductions on FPGA implementations.
The paper introduces the Von‑Neumann State‑Space Transformer (VN‑SST), a memory‑augmented Transformer that replaces the standard feed‑forward block with a low‑rank instruction bank. By decoding token‑specific operators from a low‑dimensional state‑space memory, VN‑SST achieves higher data‑efficiency and parameter‑efficiency on motor‑cortex neural‑decoding tasks and on small language‑model benchmarks. The model demonstrates that a compact instruction set can act as a control channel, improving performance without increasing accuracy through larger parameter counts.
arXiv:2605. 29548v2 Announce Type: replace Abstract: Larger models learn tasks smaller models do not.
Monotonicity has been a long-running architectural inductive bias for neural networks, motivated by tabular, scientific, and economic settings where outputs are known to respond monotonically to certain inputs. Existing approaches are MLP- or flow-based and lack per-edge functional transparency; the only Kolmogorov--Arnold Network (KAN) variant with monotonicity, MonoKAN, enforces the constraint only on a restricted parameter subset and requires a projection-style training procedure.
arXiv:2602. 22600v2 Announce Type: replace-cross Abstract: Training selects for behavior, not circuitry: many weight configurations can implement the same function.
arXiv:2607. 15217v1 Announce Type: cross Abstract: We present NeuronSoup, a neural computation architecture that replaces synchronous layer-by-layer processing with asynchronous, delay-mediated signal propagation through a pool of shared neurons.
arXiv:2606. 29519v1 Announce Type: new Abstract: Long-range learning is hard for recurrent networks trained with stochastic gradient descent, because the influence of a past input fades with the lag $\ell$, and if it fades too fast the dependence cannot be learned from finite data.
arXiv:2606. 17886v1 Announce Type: new Abstract: Monotonicity has been a long-running architectural inductive bias for neural networks, motivated by tabular, scientific, and economic settings where outputs are known to respond monotonically to certain inputs.
arXiv:2607. 14672v1 Announce Type: new Abstract: Continuous-time spiking neural networks (SNNs) provide an event-driven framework for temporal computation, computational neuroscience, and neuromorphic hardware.