arXiv AI
Increasing Width Allows Greedy Layer-wise Training to Rival End-to-End Backpropagation in Self-Supervised Learning
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jun 10
Forward-Only Convolutional Neural Networks with Learnable Channel-Class Assignment
arXiv:2606. 09928v1 Announce Type: cross Abstract: The Forward-Forward (FF) algorithm offers a biologically inspired alternative to backpropagation by replacing gradient-based credit assignment with local, forward-only objectives.
arXiv AI
Jun 15
Learning What to Predict: Downstream-Guided Task Design for Continued Pretraining
arXiv:2601. 22108v2 Announce Type: replace-cross Abstract: Continued pretraining is optimized with fixed self-supervised tasks but selected by downstream performance, creating a coarse feedback loop in which practitioners evaluate checkpoints, change data mixtures or objectives, and restart runs, while individual updates remain blind to target capabilities.
arXiv Machine Learning
Jun 24
Understanding Deep Representation Learning via Layerwise Feature Compression and Discrimination
arXiv:2311. 02960v5 Announce Type: replace Abstract: Over the past decade, deep learning has proven to be a highly effective tool for learning meaningful features from raw data.
arXiv Machine Learning
Jul 21
Effects of width-dependent model hyperparameters and $\ell_2$-regularization on the loss landscape of two-layer ReLU networks
arXiv:2607. 16720v1 Announce Type: new Abstract: Understanding deep neural networks remains a central challenge in machine learning.
arXiv AI
Sep 10
Precision at Scale: Domain-Specific Datasets On-Demand
arXiv:2407.03463v2 Announce Type: replace-cross Abstract: In the realm of self-supervised learning (SSL), conventional wisdom has gravitated towards the utility of massive, general domain datasets fo...
arXiv Machine Learning
Jun 9
Rethinking Local Learning: A Cheaper and Faster Recipe for LLM Post-Training
arXiv:2605. 04913v4 Announce Type: replace-cross Abstract: LLM post-training typically propagates task gradients through the full depth of the model.