arXiv:2408. 11629v2 Announce Type: replace Abstract: We propose a statistical-learning framework for optimization algorithms.
arXiv:2609.01493v1 Announce Type: cross Abstract: Black-Box Optimization (BBO) has found broad applications, but evolutionary algorithms and Bayesian optimization face efficiency challenges as real-w...
arXiv:2606. 08438v1 Announce Type: cross Abstract: Bayesian optimization (BO) is a widely used approach for black-box optimization that uses a Gaussian process (GP) as a surrogate and guides sequential evaluations via an acquisition function, with the ultimate goal of locating the global optimum $\mathbf{x}^{\star}$.
The paper introduces a distributionally robust method for learning hyperparameters of first‑order convex optimization algorithms. By minimizing a Wasserstein‑robust performance estimation problem over a dataset of problem instances, the approach interpolates between classical learning‑to‑optimize (L2O) and worst‑case PEP design. The authors solve the resulting problem with stochastic gradient descent, provide high‑probability risk bounds, and demonstrate that the learned algorithms outperform both worst‑case optimal and vanilla L2O baselines on logistic regression, LASSO, and linear programming tasks.
The paper introduces a curvature‑aware Expected Free Energy (EFE) acquisition function for Bayesian optimization, designed to jointly learn and optimize an underlying function. It demonstrates that, under certain assumptions, EFE reduces to familiar criteria such as Upper Confidence Bound, Lower Confidence Bound, and Expected Information Gain, and provides unbiased convergence guarantees for concave functions. Empirical results on a Van der Pol oscillator system identification task and a two‑dimensional oscillatory benchmark show that the adaptive EFE achieves competitive performance in both regret and mean squared error, outperforming typical acquisition functions that excel in only one metric.
The paper introduces a learning-based surrogate approach for stochastic optimization problems where uncertainty depends on the decision, modeled via a nonparametric regression. It constructs a surrogate that embeds iteratively updated Jacobian estimates, using an adaptive random design that focuses sampling near the current iterate to achieve dimension‑independent convergence of the Jacobian estimates. The resulting learning‑based stochastic prox‑linear (L‑SPL) algorithm demonstrates nonasymptotic convergence rates and outperforms existing methods in sample efficiency and objective value in numerical experiments.
arXiv:2505. 04757v2 Announce Type: replace Abstract: This paper introduces a novel approach to contextual stochastic optimization, integrating operations research and machine learning to address decision-making under uncertainty.
arXiv:2607. 18282v1 Announce Type: new Abstract: Bayesian Optimization is widely used for expensive black-box optimization, yet its success often depends on choosing a kernel that matches the objective's unknown structure.
arXiv:2606. 07134v1 Announce Type: new Abstract: Information-theoretic acquisition functions such as Entropy Search (ES) offer a principled exploration-exploitation framework for Bayesian optimization (BO).
The paper proposes a Bayesian decision framework for multiobjective optimization under uncertainty, focusing on maximizing the expected hypervolume over a finite set of input points. It demonstrates that gradient‑based stochastic optimization can be applied, especially when dominated points are handled carefully, and suggests using Gaussian Processes as differentiable surrogate models when direct gradients are unavailable. Additionally, the authors introduce active learning strategies via acquisition functions to build surrogate models tailored to the multiobjective problem and evaluate these strategies on simple analytical benchmarks.
arXiv:2609.36472v1 Announce Type: new Abstract: Sequential Model-Based Optimization (SMBO) traditionally relies on Bayesian or ensembling surrogates for uncertainty quantification. While historically...
arXiv:2605.28517v2 Announce Type: replace-cross Abstract: Stochastic gradient descent with momentum (SGDM) is one of the most widely used optimization algorithms in machine learning. While optimizati...
The paper presents a method that uses Constrained Bayesian Optimization (CBO) to minimize the energy consumption of machine learning models while ensuring their generalization performance stays above a specified threshold. By treating energy usage as the primary objective and performance as a constraint, the authors demonstrate that CBO can reduce training energy costs on both regression and classification tasks without sacrificing predictive accuracy.