arXiv:2605. 15435v2 Announce Type: replace Abstract: Standard deep-learning pipelines usually choose the network architecture before training and keep it fixed throughout optimization.
By Lute Lillo, Nick Cheney
arXiv:2506. 14202v4 Announce Type: replace-cross Abstract: End-to-end backpropagation requires storing activations throughout all layers, creating memory bottlenecks that limit model scalability.
By Makoto Shing, Masanori Koyama, Takuya Akiba
arXiv:2608. 04358v1 Announce Type: new Abstract: Continual learning (CL) requires models to learn tasks sequentially, yet deep neural networks often suffer from plasticity loss and poor knowledge transfer, which can impede their long-term adaptability.
By Seyed Roozbeh Razavi Rohani, Khashayar Khajavi, Wesley Chung, Mandana Samiei, Mo Chen
The paper introduces Dynamic Activity-Dependent Pruning (DADP), a biologically inspired pruning technique that uses accumulated pre‑synaptic activations and post‑synaptic error gradients to assess connection importance. Unlike traditional methods that rely on fixed layer budgets or magnitude thresholds, DADP employs a single global threshold, allowing it to dynamically allocate sparsity across a network’s depth and prune neurons and channels during training. Experiments on MLP, VGG‑16, ResNet‑18, BiLSTM‑CRF, and MiniBERT show that DADP matches or surpasses existing pruning methods, achieving 73.67% accuracy at 99% sparsity on ResNet‑18 while preserving feature diversity as measured by Shannon entropy and effective rank.
By Bhushan Deshpande
arXiv:2608.30978v1 Announce Type: new
Abstract: Modularity in deep neural networks has been proposed as a means of improving both interpretability and training by promoting disentangled representatio...
By Baptiste Rossigneux, Karim Haroun
The paper introduces the Dormant Spiking Neuron (DSN), a state-driven mechanism that allows neurons in spiking neural networks to enter a dormant state when their membrane potential falls below a negative threshold. This dynamic gating prunes redundant computations at the instance level and is trained end‑to‑end via backpropagation through time with surrogate gradients. Experiments on static and neuromorphic datasets, including ImageNet, show that DSN reduces computational energy by 40% to over 70% and improves baseline SNN accuracy by 1.34%, outperforming state‑of‑the‑art static pruning methods.
By Ruichen Ma, Yicai Chen, Pujun Zhou, Yue Zuo, Chongyang Li, Shuang Liu, Shaogang Hu, Guanchao Qiao