The study investigates whether sensory-aligned receptive fields provide computational benefits beyond mere resource efficiency in recurrent networks of Expressive Leaky Memory neurons. Across auditory and event-based visual classification tasks, receptive fields aligned with task-relevant sensory coordinates improve test accuracy compared to budget-matched random fields, but this advantage disappears when coordinates are scrambled or irrelevant. The benefit diminishes as neuronal expressivity increases, and generic synaptic sparsity regularization only partially recovers performance, indicating that structured receptive fields act as a computational prior beyond sparsity alone.
Spiking neural networks (SNNs) are promoted as an energy-efficient substrate because sparse, event-driven activity replaces dense multiply-accumulates with cheap accumulates. We argue the energy dividend of sparsity is not a property of SNNs but of the task.
arXiv:2603. 01568v2 Announce Type: replace Abstract: Efficient coding theory predicts that biological perceptual systems compress sensory input optimally under resource constraints, with the systematic structure of errors reflecting the geometry of that compression.
arXiv:2609.36391v1 Announce Type: cross Abstract: An ongoing challenge in sensory neuroscience is to characterize the feature dimensions encoded by cortical populations. Recent approaches probe featu...
The paper argues that a convolutional sequence labeler’s receptive field is not the sole source of context. When a normalization layer computes statistics across the entire sequence at inference, it creates a sequence‑spanning path that supplies global context, effectively replacing the need for a larger receptive field. Experiments on synthetic labeling tasks, simulated genomes, and real 1000 Genomes haplotypes show that this global summary can match or exceed the performance of larger receptive fields, and that ablating receptive‑field‑enlarging blocks overestimates their importance.
arXiv:2609.24379v1 Announce Type: cross Abstract: Mechanistic interpretability of vision transformers seeks to decompose model computation into human-readable units, but learned representations entan...
arXiv:2607. 26648v1 Announce Type: cross Abstract: Spiking neural networks (SNNs) are promoted as an energy-efficient substrate because sparse, event-driven activity replaces dense multiply-accumulates with cheap accumulates.
arXiv:2503. 21796v2 Announce Type: replace-cross Abstract: Self-supervised learning has become an increasingly important paradigm in the domain of machine intelligence.
The paper argues that a convolutional sequence labeler’s receptive field is not the sole source of context; a normalization layer that computes statistics across the entire sequence at inference creates a global context path that bypasses the receptive field. By analyzing the Jacobian of the normalization layer, the authors show that this path provides almost all the benefit of a larger receptive field, especially when labels appear in long runs. Experiments on synthetic data, simulated genomes, and real 1000 Genomes haplotypes demonstrate that closing this path can multiply the value of enlarging the receptive field by up to an order of magnitude, and that ablating receptive‑field‑enlarging blocks overestimates their contribution due to this hidden path.
arXiv:2606. 00073v1 Announce Type: cross Abstract: We investigate how internal representations emerge across hierarchical processing systems by introducing a neuroscience-inspired framework for analyzing deep spiking neural networks (SNN) through the lens of functional connectivity.
Mechanistic interpretability of vision transformers seeks to decompose model computation into human-readable units, but learned representations entangle many concepts in each neuron. Feature superposi...
arXiv:2607. 16295v1 Announce Type: cross Abstract: Mechanistic interpretability has made significant strides in understanding neural network representations, with sparse dictionary learning (SDL) methods, most prominently sparse autoencoders, as a central paradigm.
arXiv:2512. 11000v2 Announce Type: replace-cross Abstract: Representations pervade our daily experience, from letters representing sounds to bit strings encoding digital files.