arXiv AI
Scaling Laws and Tradeoffs in Recurrent Networks of Expressive Neurons
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
Hugging Face Trending Papers
Jul 29
The Sparsity Ceiling: Where Spiking Networks Can and Cannot Trade Activity for Energy
Spiking neural networks (SNNs) are promoted as an energy-efficient substrate because sparse, event-driven activity replaces dense multiply-accumulates with cheap accumulates. We argue the energy dividend of sparsity is not a property of SNNs but of the task.
arXiv Machine Learning
Jul 30
The Sparsity Ceiling: Where Spiking Networks Can and Cannot Trade Activity for Energy
arXiv:2607. 26648v1 Announce Type: cross Abstract: Spiking neural networks (SNNs) are promoted as an energy-efficient substrate because sparse, event-driven activity replaces dense multiply-accumulates with cheap accumulates.
arXiv Computation and Language
Sep 15
URCHIN: A Horizontal Spiking Language Model for Data-Constrained Pretraining
arXiv:2609.13899v1 Announce Type: cross Abstract: The BabyLM challenge measures how much language a model can learn from developmentally-plausible, child-scale data rather than internet-scale corpora...
arXiv AI
Oct 12
NeuronSpark: Selective Spiking State Space Models with Adaptive Computation for Language Modeling
NeuronSpark is a self‑attention‑free language model that uses input‑dependent spiking membrane dynamics, allowing each neuron’s decay rate, gain, and firing threshold to adapt to the current token. This selective spiking approach enables billion‑parameter pretraining from random initialization without relying on backbone transfer, conversion, or distillation, achieving competitive zero‑shot performance and a fast–slow hierarchy. On a 1.1B‑parameter, 2.1B‑token training run, NeuronSpark attains a 40.0 eight‑task zero‑shot average, surpasses comparable‑budget Pythia/Mamba checkpoints, and demonstrates significant energy and cycle reductions on FPGA implementations.
arXiv Machine Learning
Aug 27
The Von-Neumann State-Space Transformer for neural decoding
The paper introduces the Von‑Neumann State‑Space Transformer (VN‑SST), a memory‑augmented Transformer that replaces the standard feed‑forward block with a low‑rank instruction bank. By decoding token‑specific operators from a low‑dimensional state‑space memory, VN‑SST achieves higher data‑efficiency and parameter‑efficiency on motor‑cortex neural‑decoding tasks and on small language‑model benchmarks. The model demonstrates that a compact instruction set can act as a control channel, improving performance without increasing accuracy through larger parameter counts.
arXiv Machine Learning
Jun 2
Why Larger Models Learn More: Effects of Capacity, Interference, and Rare-Task Retention
arXiv:2605. 29548v2 Announce Type: replace Abstract: Larger models learn tasks smaller models do not.