arXiv Machine Learning
Jun 9
LEAF: A Learning-Enabled ADMM Framework for Accelerated Convex Optimization
arXiv:2606. 08993v1 Announce Type: new Abstract: We propose LEAF, a learning-enabled ADMM framework for accelerated convex optimization.
Learning to Optimize by Differentiable Programming
arXiv:2601. 16510v3 Announce Type: replace-cross Abstract: Solving massive-scale optimization problems requires scalable first-order methods with low per-iteration cost.
Related stories
arXiv Machine Learning
Jun 16
A Fully First-Order Layer for Differentiable Optimization
arXiv:2512. 02494v2 Announce Type: replace Abstract: Differentiable optimization layers enable learning systems to make decisions by solving embedded optimization problems.
arXiv AI
Jun 3
Introduction to optimization methods for training SciML models
arXiv:2601. 10222v2 Announce Type: replace-cross Abstract: Optimization is central to both modern machine learning (ML) and scientific machine learning (SciML), yet the structure of the underlying optimization problems differs substantially across these domains.
arXiv Machine Learning
Jul 27
Energy Manifold Natural Gradient Descent: Riemannian Optimization for Neural PDE Solvers
arXiv:2607. 22004v1 Announce Type: new Abstract: Energy natural gradient descent (ENGD) aligns parameter updates with the curvature of an underlying function-space energy, but existing formulations assume an unconstrained Euclidean parameter domain.
arXiv Machine Learning
Jun 29
Mosaic: A Benchmark Suite for Differentiable Physics Solvers
arXiv:2606. 27895v1 Announce Type: cross Abstract: Differentiable partial differential equation (PDE) solvers underpin solver-in-the-loop ML training, gradient-based optimal control, and inverse problems, yet the practical cost of obtaining correct, usable gradients from a given solver on a given problem is largely undocumented.
arXiv Machine Learning
Jun 30
Extrapolating from Regularised Solutions for Solving Ill-Conditioned Linear Systems in Machine Learning
arXiv:2606. 30328v1 Announce Type: cross Abstract: Rapid prototyping of algorithms is a critical step in modern machine learning.
arXiv Machine Learning
Jun 16
Self-Supervised Learning of Iterative Solvers for Constrained Optimization
arXiv:2409. 08066v3 Announce Type: replace Abstract: The real-time solution of parametric optimization problems is critical for applications that demand high accuracy under tight real-time constraints, such as model predictive control.
arXiv Machine Learning
Jun 9
Parameter Tuning with Generalization Guarantees for GPU-Accelerated Linear Programming
arXiv:2606. 08638v1 Announce Type: cross Abstract: Recent research has developed practical, parallelizable first-order methods for large scale linear programming, but performance is highly dependent on hyperparameter selection.
arXiv Machine Learning
Jul 27
Complexity Bounds and Approaches to Learning Projected Gradient Descent Solver Iterates
arXiv:2607. 22467v1 Announce Type: new Abstract: Data scarcity poses a fundamental challenge in training generative models to produce initial guesses for parametric optimization problems that are otherwise numerically expensive to solve.
arXiv Machine Learning
Jul 7
Learning rate adaptive stochastic gradient descent optimization methods: numerical simulations for deep learning methods for partial differential equations and convergence analyses
arXiv:2406. 14340v2 Announce Type: replace-cross Abstract: The standard stochastic gradient descent (SGD) optimization method, as well as adaptive methods such as the Adam optimizer fail to converge if the learning rates do not converge to zero (particularly, in the situation of constant learning rates).
arXiv Machine Learning
Aug 4
Simplified Quadratic Gradient: A Unified Framework Bridging Gradient Descent and Newton-Type Methods by Synthesizing Hessians and Gradients
arXiv:2209. 03282v5 Announce Type: replace-cross Abstract: Accelerating the convergence of second-order optimization, particularly Newton-type methods, remains a pivotal challenge in algorithmic research.
arXiv Machine Learning
Jun 16
Functional Gradient Descent with Adaptive Representations
arXiv:2606. 16926v1 Announce Type: cross Abstract: Functional optimization problems are typically solved by optimizing the parameters of a fixed representation, such as a neural network, resulting in highly nonconvex losses that complicate both training and theoretical analysis.