arXiv Machine Learning June 16, 2026 By Daniel Csillag, Rodrigo Schuller, Pedro Dall'Antonia, Leonidas Guibas, Luiz Velho, Tiago Novello

Functional Gradient Descent with Adaptive Representations

Read the original on arXiv Machine Learning →

arXiv:2606. 16926v1 Announce Type: cross Abstract: Functional optimization problems are typically solved by optimizing the parameters of a fixed representation, such as a neural network, resulting in highly nonconvex losses that complicate both training and theoretical analysis.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

computer-vision

arXiv Machine Learning

Aug 4

Simplified Quadratic Gradient: A Unified Framework Bridging Gradient Descent and Newton-Type Methods by Synthesizing Hessians and Gradients

arXiv:2209. 03282v5 Announce Type: replace-cross Abstract: Accelerating the convergence of second-order optimization, particularly Newton-type methods, remains a pivotal challenge in algorithmic research.

By John Chiang

arXiv Machine Learning

6d ago

A New First-Order Meta-Learning Algorithm with Convergence Guarantees

arXiv:2409. 03682v2 Announce Type: replace Abstract: Learning new tasks by leveraging prior experience is a fundamental trait of intelligent systems.

By El Mahdi Chayti, Martin Jaggi

llms efficiency safety

arXiv Machine Learning

Jul 30

Adaptive Gradient-Based Methods for a Broader Class of Optimization Problems under Performative Prediction

arXiv:2607. 26562v1 Announce Type: cross Abstract: We study optimization under performative prediction, where deploying a model affects the future data distribution.

By Hiroki Hamaguchi, Yuya Hikima, Hiroshi Sawada, Akiko Takeda

arXiv Machine Learning

Aug 11

Generalized Convexity and Smoothness via Conjugate Duality: Optimization Theory for Deep Neural Networks

arXiv:2608. 09523v1 Announce Type: new Abstract: Deep neural network (DNN) training with stochastic gradient descent (SGD) and its variants achieves strong empirical performance, yet classical optimization theory does not fully explain this success.

By Binchuan Qi

safety

arXiv Machine Learning

Jun 5

Learning to optimize with guarantees: a complete characterization of linearly convergent algorithms

arXiv:2508. 00775v2 Announce Type: replace-cross Abstract: The design of many classical optimization algorithms is driven by the certification of linear convergence rates over classes of optimization problems.

By Andrea Martin, Ian R. Manchester, Luca Furieri

rag

arXiv Machine Learning

Jun 16

A Fully First-Order Layer for Differentiable Optimization

arXiv:2512. 02494v2 Announce Type: replace Abstract: Differentiable optimization layers enable learning systems to make decisions by solving embedded optimization problems.

By Zihao Zhao, Kai-Chia Mo, Shing-Hei Ho, Brandon Amos, Kai Wang

Related stories

Simplified Quadratic Gradient: A Unified Framework Bridging Gradient Descent and Newton-Type Methods by Synthesizing Hessians and Gradients

A New First-Order Meta-Learning Algorithm with Convergence Guarantees

Adaptive Gradient-Based Methods for a Broader Class of Optimization Problems under Performative Prediction

Generalized Convexity and Smoothness via Conjugate Duality: Optimization Theory for Deep Neural Networks

Learning to optimize with guarantees: a complete characterization of linearly convergent algorithms

A Fully First-Order Layer for Differentiable Optimization