arXiv AI

Increasing Width Allows Greedy Layer-wise Training to Rival End-to-End Backpropagation in Self-Supervised Learning

arXiv AI

Jun 10

Forward-Only Convolutional Neural Networks with Learnable Channel-Class Assignment

arXiv:2606. 09928v1 Announce Type: cross Abstract: The Forward-Forward (FF) algorithm offers a biologically inspired alternative to backpropagation by replacing gradient-based credit assignment with local, forward-only objectives.

By Mohammadnavid Ghader, Saeed Reza Kheradpisheh, Bahar Farahani, Mahmood Fazlali

benchmarks

arXiv AI

Jun 15

Learning What to Predict: Downstream-Guided Task Design for Continued Pretraining

arXiv:2601. 22108v2 Announce Type: replace-cross Abstract: Continued pretraining is optimized with fixed self-supervised tasks but selected by downstream performance, creating a coarse feedback loop in which practitioners evaluate checkpoints, change data mixtures or objectives, and restart runs, while individual updates remain blind to target capabilities.

By Shuqi Ke, Giulia Fanti

computer-vision

arXiv Machine Learning

Jun 24

Understanding Deep Representation Learning via Layerwise Feature Compression and Discrimination

arXiv:2311. 02960v5 Announce Type: replace Abstract: Over the past decade, deep learning has proven to be a highly effective tool for learning meaningful features from raw data.

By Peng Wang, Xiao Li, Can Yaras, Zhihui Zhu, Laura Balzano, Wei Hu, Qing Qu

fine-tuning

arXiv Machine Learning

Jul 21

Effects of width-dependent model hyperparameters and $\ell_2$-regularization on the loss landscape of two-layer ReLU networks

arXiv:2607. 16720v1 Announce Type: new Abstract: Understanding deep neural networks remains a central challenge in machine learning.

By Haruka Eshima, Makoto Yamada

arXiv AI

Sep 10

Precision at Scale: Domain-Specific Datasets On-Demand

arXiv:2407.03463v2 Announce Type: replace-cross Abstract: In the realm of self-supervised learning (SSL), conventional wisdom has gravitated towards the utility of massive, general domain datasets fo...

By Jes\'us M Rodr\'iguez-de-Vera, Imanol G Estepa, Ignacio Saras\'ua, Bhalaji Nagarajan, Petia Radeva

llms agents benchmarks

arXiv Machine Learning

Jun 9

Rethinking Local Learning: A Cheaper and Faster Recipe for LLM Post-Training

arXiv:2605. 04913v4 Announce Type: replace-cross Abstract: LLM post-training typically propagates task gradients through the full depth of the model.

By Hengyu Shi, Tianyang Han, Peizhe Wang, Zhiling Wang, Xu Yang, Junhao Su

llms

arXiv Machine Learning

Jul 28

Self-Distillation of Hidden Layers for Self-Supervised Representation Learning

arXiv:2603. 15553v2 Announce Type: replace-cross Abstract: The landscape of self-supervised learning (SSL) is currently dominated by generative approaches (e.

By Scott C. Lowe, Anthony Fuller, Sageev Oore, Evan Shelhamer, Graham W. Taylor

rag computer-vision efficiency

arXiv Statistics ML

2d ago

Transferable Graph Metanetworks

arXiv:2610.00420v1 Announce Type: new Abstract: A weight space network (or metanetwork) takes the weights of another neural network as input and predicts properties of it. Most prior work trains such...

By Yuxin Ma, Adir Dayan, Yam Eitan, Haggai Maron, Soledad Villar

arXiv Machine Learning

Jun 3

IdEst: Assessing Self-Supervised Learning Representations via Intrinsic Dimension

arXiv:2606. 03338v1 Announce Type: new Abstract: Self-supervised learning (SSL) has emerged as a powerful paradigm for learning meaningful representations from unlabeled data.

By Julie Mordacq, Vicky Kalogeiton, Steve Oudot

arXiv Machine Learning

Jul 24

Statistical physics of deep learning: Optimal learning of a multi-layer perceptron near interpolation

arXiv:2510. 24616v4 Announce Type: replace-cross Abstract: For four decades statistical physics has been providing a framework to analyse neural networks.

By Jean Barbier, Francesco Camilli, Minh-Toan Nguyen, Mauro Pastore, Rudy Skerk

rag

arXiv Machine Learning

Jun 11

On the Stability of Growth in Structural Plasticity

arXiv:2605. 15435v2 Announce Type: replace Abstract: Standard deep-learning pipelines usually choose the network architecture before training and keep it fixed throughout optimization.

By Lute Lillo, Nick Cheney

efficiency benchmarks

arXiv Machine Learning

Jul 9

Efficient Long-Horizon Learning for Learned Optimization

arXiv:2607. 06772v1 Announce Type: new Abstract: Learned optimization aims to improve upon hand-designed optimizers (e.

By Xiaolong Huang, Benjamin Th\'erien, James Harrison, Eugene Belilovsky

llms computer-vision

Related stories