arXiv Machine Learning
Jul 28
Greedy dynamical meta-learning
arXiv:2607. 23925v1 Announce Type: new Abstract: Gradient descent scales well to large models, but becomes unstable over long time horizons.
On first-order meta-learning algorithms
Related stories
Hugging Face Trending Papers
Jul 27
Greedy dynamical meta-learning
Gradient descent scales well to large models, but becomes unstable over long time horizons. Gradient-free optimizers can scale to arbitrary timespans, but are hobbled by high dimensions.
OpenAI Blog
Mar 7, 2018
Reptile: A scalable meta-learning algorithm
We’ve developed a simple meta-learning algorithm called Reptile which works by repeatedly sampling a task, performing stochastic gradient descent on it, and updating the initial parameters towards the final parameters learned on that task. Reptile is the application of the Shortest Descent algorithm to the meta-learning setting, and is mathematically similar to first-order MAML (which is a version of the well-known MAML algorithm) that only needs black-box access to an optimizer such as SGD or Adam, with similar computational efficiency and performance.
arXiv Machine Learning
6d ago
A New First-Order Meta-Learning Algorithm with Convergence Guarantees
arXiv:2409. 03682v2 Announce Type: replace Abstract: Learning new tasks by leveraging prior experience is a fundamental trait of intelligent systems.
Hugging Face Trending Papers
Jun 1
Provable Data Scaling Law for Meta Learning via Complexity Minimization
Pre-training has become a fundamental paradigm in modern machine learning, with one of its key empirical benefits being reduced downstream sample complexity as the scale of pre-training data increases. However, existing theoretical frameworks for pre-training do not fully explain this phenomenon.
arXiv Machine Learning
Jun 2
Provable Data Scaling Law for Meta Learning via Complexity Minimization
arXiv:2606. 02008v1 Announce Type: cross Abstract: Pre-training has become a fundamental paradigm in modern machine learning, with one of its key empirical benefits being reduced downstream sample complexity as the scale of pre-training data increases.
arXiv Machine Learning
Jul 14
Metadata-Free Meta-Reweighted Direct Preference Optimization under Noisy Preference Labels
arXiv:2607. 09796v1 Announce Type: new Abstract: Direct Preference Optimization (DPO) has become an important method for aligning large language models (LLMs) with human preferences because it removes the need for explicit reward modeling and reinforcement learning optimization.
arXiv Machine Learning
Jun 5
Efficiently Escaping Saddle Points under Generalized Smoothness via Self-Bounding Regularity
arXiv:2503. 04712v3 Announce Type: replace-cross Abstract: We study the optimization of non-convex functions that are not necessarily smooth (gradient and/or Hessian are Lipschitz) using first order methods.
Hugging Face Trending Papers
Jul 7
Efficient Long-Horizon Learning for Learned Optimization
Learned optimization aims to improve upon hand-designed optimizers (e. g.
arXiv Machine Learning
5d ago
Comment on "Modeling rapid language learning by distilling Bayesian priors into artificial neural networks"
arXiv:2608. 12974v1 Announce Type: new Abstract: McCoy & Griffiths (2025, henceforth M&G) suggest that a Bayesian prior can be distilled into Artificial Neural Networks (ANNs) through Model-Agnostic Meta-Learning (MAML, Finn et al.
arXiv AI
Jun 2
On the Difficulty of Learning a Meta-network for Training Data Selection
arXiv:2606. 00571v1 Announce Type: cross Abstract: Synthetic data are increasingly used to train neural networks, yet distributional mismatch with real data limits their effectiveness when used indiscriminately.
arXiv AI
Jul 8
Universal Algorithm-Implicit Learning
arXiv:2602. 14761v2 Announce Type: replace-cross Abstract: Current meta-learning methods are constrained to narrow task distributions with fixed feature and label spaces, limiting applicability.