arXiv Machine Learning
Jul 9
Efficient Long-Horizon Learning for Learned Optimization
arXiv:2607. 06772v1 Announce Type: new Abstract: Learned optimization aims to improve upon hand-designed optimizers (e.
On the Stability of Growth in Structural Plasticity
arXiv:2605. 15435v2 Announce Type: replace Abstract: Standard deep-learning pipelines usually choose the network architecture before training and keep it fixed throughout optimization.
Related stories
arXiv Machine Learning
Jul 28
Self-Motivated Growing Neural Network for Adaptive Architecture via Local Structural Plasticity
arXiv:2512. 12713v2 Announce Type: replace-cross Abstract: Control policies are often implemented with fixed-capacity multilayer perceptrons trained by backpropagation, which require architecture selection in advance and cannot adapt their capacity during learning.
arXiv AI
Jul 22
Soft-TransFormers for Continual Learning
arXiv:2411. 16073v4 Announce Type: replace-cross Abstract: Inspired by the Well-initialized Lottery Ticket Hypothesis (WLTH), we introduce Soft-TransFormers (Soft-TF), a continual learning framework that adapts a frozen pre-trained Transformer through task-specific soft subnetworks: real-valued multiplicative masks over the query, key, value, and output projections of selected self-attention layers.
arXiv Machine Learning
Jun 30
HiAP: A Multi-Granular Stochastic Auto-Pruning Framework for Vision Transformers
arXiv:2603. 12222v2 Announce Type: replace-cross Abstract: Vision Transformers require significant computational resources and memory bandwidth, severely limiting their deployment on resource-constraint hardware.
Hugging Face Trending Papers
Jul 7
Efficient Long-Horizon Learning for Learned Optimization
Learned optimization aims to improve upon hand-designed optimizers (e. g.
arXiv Machine Learning
Aug 4
Plasticity of Growing and Elastic Neural Networks in Online Continual Learning
arXiv:2608. 01475v1 Announce Type: new Abstract: Neural networks that can grow or both grow and shrink during learning, referred to as growing neural networks and elastic neural networks, respectively, have recently been explored in offline continual learning with a particular focus on catastrophic forgetting.
arXiv Machine Learning
Jul 20
Learning Faster without Deeper Networks: A*-Inspired Batch Selection for Efficient CNN Training
arXiv:2607. 15745v1 Announce Type: new Abstract: Common practice when training Convolutional Neural Networks (CNNs) is to use randomly shuffled mini-batches.
arXiv Machine Learning
Jul 16
Local Redundancy: An Information-Theoretic Measure of Plasticity from Synthetic Memorization
arXiv:2607. 13432v1 Announce Type: new Abstract: Plasticity -- a neural network's ability to adapt to new tasks -- is critical for continual and transfer learning.
arXiv AI
Jul 29
CIFNet: An Analytic Neural Learning Framework for Efficient and Calibrated Class-Incremental Learning
arXiv:2509. 11285v2 Announce Type: replace-cross Abstract: Class-Incremental Learning (CIL) in deep neural networks is conventionally framed as an iterative gradient-based optimization problem, incurring high computational cost, hyperparameter sensitivity, and risk of catastrophic forgetting.
arXiv AI
Jul 17
NORACL: Neurogenesis for Oracle-free Resource-Adaptive Continual Learning
arXiv:2604. 27031v2 Announce Type: replace-cross Abstract: In a continual learning setting, we require a model to be plastic enough to learn a new task and stable enough to not disturb previously learned capabilities.
arXiv AI
Jun 16
Scaling Adaptive Depth with Norm-Agnostic Residual Networks
arXiv:2606. 16112v1 Announce Type: cross Abstract: Residual architectures are ubiquitous in deep learning, but they suffer from a subtle structural limitation: the norm of the residual stream can grow rapidly with depth.
arXiv AI
Jun 17
PLATE: Plasticity-Tunable Efficient Adapters for Geometry-Aware Continual Learning
arXiv:2602. 03846v2 Announce Type: replace-cross Abstract: We develop a continual learning method for pretrained models that \emph{requires no access to old-task data}, addressing a practical barrier in foundation model adaptation where pretraining distributions are often unavailable.