← Back to all news
arXiv Machine Learning By Mohsen Amidzadeh, Lauri Viitasaari, Mario Di Francesco

Non-Parametric Optimization for Scalable Learning in Stochastic Decision Problems

Read the original on arXiv Machine Learning →

The paper introduces a stochastic variational framework using Malliavin calculus to derive non‑parametric optimality conditions for time‑varying stochastic optimization problems. It proposes the Stochastic Path Follower (SPF), a deep‑learning algorithm that remains scalable regardless of parameter‑space dimensionality. Experiments on least‑squares recovery and logistic regression under distribution drift demonstrate that SPF outperforms existing learning‑based and gradient‑based methods in both performance and scalability.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Statistics ML
Sep 11

Learning-Based Surrogate Method for Stochastic Optimization under Decision-Dependent Uncertainty with Adaptive Random Designs

The paper introduces a learning-based surrogate approach for stochastic optimization problems where uncertainty depends on the decision, modeled via a nonparametric regression. It constructs a surrogate that embeds iteratively updated Jacobian estimates, using an adaptive random design that focuses sampling near the current iterate to achieve dimension‑independent convergence of the Jacobian estimates. The resulting learning‑based stochastic prox‑linear (L‑SPL) algorithm demonstrates nonasymptotic convergence rates and outperforms existing methods in sample efficiency and objective value in numerical experiments.

By Boyang Shen, Junyi Liu
ragbenchmarks
More like this →
arXiv Machine Learning
Jun 2

Safeguarded Stochastic Polyak Step Sizes for Non-smooth Optimization: Robust Performance Without Small (Sub)Gradients

arXiv:2512. 02342v3 Announce Type: replace-cross Abstract: The stochastic Polyak step size (SPS) has proven to be a promising choice for stochastic gradient descent (SGD), delivering competitive performance relative to state-of-the-art methods on smooth convex and non-convex optimization problems, including deep neural network training.

By Dimitris Oikonomou, Nicolas Loizou
benchmarks
More like this →
arXiv Machine Learning
Jul 7

Learning rate adaptive stochastic gradient descent optimization methods: numerical simulations for deep learning methods for partial differential equations and convergence analyses

arXiv:2406. 14340v2 Announce Type: replace-cross Abstract: The standard stochastic gradient descent (SGD) optimization method, as well as adaptive methods such as the Adam optimizer fail to converge if the learning rates do not converge to zero (particularly, in the situation of constant learning rates).

By Steffen Dereich, Arnulf Jentzen, Adrian Riekert
More like this →