arXiv Machine Learning
Jul 28
Greedy dynamical meta-learning
arXiv:2607. 23925v1 Announce Type: new Abstract: Gradient descent scales well to large models, but becomes unstable over long time horizons.
On first-order meta-learning algorithms
Related stories
Hugging Face Trending Papers
Jul 27
Greedy dynamical meta-learning
Gradient descent scales well to large models, but becomes unstable over long time horizons. Gradient-free optimizers can scale to arbitrary timespans, but are hobbled by high dimensions.
OpenAI Blog
Mar 7, 2018
Reptile: A scalable meta-learning algorithm
We’ve developed a simple meta-learning algorithm called Reptile which works by repeatedly sampling a task, performing stochastic gradient descent on it, and updating the initial parameters towards the final parameters learned on that task. Reptile is the application of the Shortest Descent algorithm to the meta-learning setting, and is mathematically similar to first-order MAML (which is a version of the well-known MAML algorithm) that only needs black-box access to an optimizer such as SGD or Adam, with similar computational efficiency and performance.
arXiv Machine Learning
Aug 13
A New First-Order Meta-Learning Algorithm with Convergence Guarantees
arXiv:2409. 03682v2 Announce Type: replace Abstract: Learning new tasks by leveraging prior experience is a fundamental trait of intelligent systems.
Hugging Face Trending Papers
Jun 1
Provable Data Scaling Law for Meta Learning via Complexity Minimization
Pre-training has become a fundamental paradigm in modern machine learning, with one of its key empirical benefits being reduced downstream sample complexity as the scale of pre-training data increases. However, existing theoretical frameworks for pre-training do not fully explain this phenomenon.
arXiv Machine Learning
Jun 2
Provable Data Scaling Law for Meta Learning via Complexity Minimization
arXiv:2606. 02008v1 Announce Type: cross Abstract: Pre-training has become a fundamental paradigm in modern machine learning, with one of its key empirical benefits being reduced downstream sample complexity as the scale of pre-training data increases.
arXiv Machine Learning
Jul 14
Metadata-Free Meta-Reweighted Direct Preference Optimization under Noisy Preference Labels
arXiv:2607. 09796v1 Announce Type: new Abstract: Direct Preference Optimization (DPO) has become an important method for aligning large language models (LLMs) with human preferences because it removes the need for explicit reward modeling and reinforcement learning optimization.
arXiv Machine Learning
Aug 26
TLXML: Task-Level Explanation of Meta-Learning via Influence Functions
arXiv:2501.14271v4 Announce Type: replace Abstract: Meta-learning enables models to rapidly adapt to new tasks by leveraging prior experience, but its adaptation mechanisms remain opaque, especially...
arXiv Machine Learning
4d ago
Meta-learning accelerates detector design optimization
arXiv:2609.35827v1 Announce Type: cross Abstract: The quality of a detector design is ultimately determined by the quality of the inference it enables, that is, by the accuracy with which the quantit...
arXiv Machine Learning
Jun 5
Efficiently Escaping Saddle Points under Generalized Smoothness via Self-Bounding Regularity
arXiv:2503. 04712v3 Announce Type: replace-cross Abstract: We study the optimization of non-convex functions that are not necessarily smooth (gradient and/or Hessian are Lipschitz) using first order methods.
Hugging Face Trending Papers
Jul 7
Efficient Long-Horizon Learning for Learned Optimization
Learned optimization aims to improve upon hand-designed optimizers (e. g.
arXiv Machine Learning
Sep 16
Federated stochastic bilevel optimization with fully first-order gradients
arXiv:2609.16350v1 Announce Type: new Abstract: Federated stochastic bilevel optimization has been actively studied in recent years due to its widespread applications in machine learning. However, mo...