← Back to all news
arXiv Machine Learning By Shokhrukh Ibragimov, Arnulf Jentzen

Unified convergence analysis for gradient descent optimization methods in the training of deep neural networks

Read the original on arXiv Machine Learning →

arXiv:2607. 04233v1 Announce Type: cross Abstract: Gradient based optimization methods are nowadays the methods of choice for training deep neural networks (DNNs) in artificial intelligence (AI) systems.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Sep 2

Uniform a priori bounds and error analysis for the Adam stochastic gradient descent optimization method

The paper establishes uniform a priori bounds for the Adam optimizer, enabling an unconditional error analysis for a broad class of strongly convex stochastic optimization problems. Prior analyses were conditional, assuming Adam remained bounded, whereas this work removes that assumption. The results provide a rigorous foundation for Adam’s performance in training deep neural networks and other convex optimization tasks.

By Steffen Dereich, Thang Do, Arnulf Jentzen
More like this →
arXiv Machine Learning
Jul 7

Learning rate adaptive stochastic gradient descent optimization methods: numerical simulations for deep learning methods for partial differential equations and convergence analyses

arXiv:2406. 14340v2 Announce Type: replace-cross Abstract: The standard stochastic gradient descent (SGD) optimization method, as well as adaptive methods such as the Adam optimizer fail to converge if the learning rates do not converge to zero (particularly, in the situation of constant learning rates).

By Steffen Dereich, Arnulf Jentzen, Adrian Riekert
More like this →