← Back to all news
arXiv Machine Learning

Multi-Objective Hyperparameter Search via Damped Gauss--Newton Optimization

Related stories

arXiv Machine Learning
Sep 24

tidyHEBO: Robust General-Purpose Bayesian Optimization with Model-Consistent Warping and Pareto Search

tidyHEBO is a BoTorch-native Bayesian optimization tool that jointly applies Yeo-Johnson output warping to a Gaussian‑process surrogate, evaluates acquisition functions on the original objective scale, and conducts constrained cumulative Pareto search across multiple acquisition criteria. Using only default settings, it outperformed other methods on the Olympus benchmark and performed strongly on synthetic, Needle‑in‑a‑Haystack, and Bayesmark tasks, while adaptive batching offered a trade‑off between parallelization and optimization quality. These results position tidyHEBO as a robust, reproducible optimizer suitable for diverse practical problems, including scientific applications and hyperparameter tuning.

By L. A. Zhukov, E. V. Shaburova, D. V. Antonets
benchmarks
More like this →
arXiv Machine Learning
1d ago

Calibrating Prediction Timeliness Through Multi-Objective Hyperparameter Optimization for Remaining Useful Life Prediction

The paper investigates how treating the optimization objective as a design variable can improve Remaining Useful Life (RUL) prediction in predictive maintenance. Five model architectures are compared under single‑objective and multi‑objective hyperparameter optimization, with the latter using NSGA‑II and Entropy‑CRITIC weighting to balance accuracy and prediction timeliness. Results on NASA C‑MAPSS and BackBlaze datasets show that multi‑objective optimization reduces directional imbalance in predictions and can alter model rankings, highlighting the importance of objective choice in RUL modeling.

By Tugrul Cabir Hakyemez, Ener Uras Gokhan
llmsbenchmarks
More like this →
arXiv Machine Learning
Sep 11

Statistically Valid Post-Training Hyperparameter Selection: From Tuning to Guarantees

The paper introduces a statistical framework for post‑training hyperparameter selection, emphasizing the learn‑then‑test (LTT) paradigm. It treats hyperparameter tuning as a multiple hypothesis testing problem over a candidate set, enabling the selection of hyperparameters that meet specified reliability constraints such as risk bounds or information‑theoretic limits. The framework provides finite‑sample control of error probabilities using p‑values, e‑values, and concentration inequalities derived from first principles.

By Amirmohammad Farzaneh, Osvaldo Simeone
safety
More like this →
arXiv Machine Learning
Sep 4

No-Regret Bayesian Optimization with Finite-Library Input-Warped Kernels

The paper introduces Finite-Library Input-Warped Bayesian Optimization (FLIWBO), a method that selects input warps from a finite library to adapt the geometry used by Gaussian‑process Bayesian optimization. FLIWBO maintains high‑probability convergence guarantees while improving sample efficiency on problems where raw coordinates poorly match the objective’s geometry, such as log‑scaled hyperparameters or localized peaks. Experiments on synthetic benchmarks, Fashion‑MNIST hyperparameter tuning, and a 20‑dimensional multi‑agent system design demonstrate that FLIWBO‑UCB outperforms raw‑coordinate GP‑UCB and other methods with regret guarantees, especially under misspecified geometry.

By Edvin Ketabati Augustinsson, Robert A. Bridges
agentsbenchmarks
More like this →
arXiv Machine Learning
Sep 15

A family of spectral conjugate gradient algorithms derived by least-squares approximations based on a modified quasi--Newton update with application to a revised robust binary classification model

arXiv:2609.13526v1 Announce Type: cross Abstract: We develop a spectral three-term modification of the classic Hestenes--Stiefel conjugate gradient algorithm, preserving its anti-jamming characterist...

By Saman Babaie-Kafaki, Maryam Khoshsimaye-Bargard, Ahmad Mousavi
More like this →