arXiv AI

Artificial Neural Networks as Surrogate Models in Black Box Optimization

arXiv Machine Learning
Sep 17

Benchmarking Tabular Foundation Models as Surrogates in Expensive Evolutionary Optimization

The paper evaluates the Tabular Prior-data Fitted Network (TabPFN) as a surrogate model in surrogate‑assisted evolutionary algorithms (SAEAs) for expensive optimization problems. Through extensive experiments in both offline and online settings across a range of problem types—including single‑objective, multi‑objective, constrained, combinatorial, mixed‑variable, and engineering tasks—the study finds that TabPFN’s effectiveness varies strongly with the problem characteristics. The authors conclude that TabPFN should be used selectively, with customized model management and algorithm design tailored to data availability, landscape complexity, and search‑space properties.

By Lu Han, Jin Wang, Yuchen Li, Haoran Gu, Shulei Liu, Ziyang Shi, Wenao Lu, Handing Wang
arXiv AI
Jun 30

Bilevel Optimization for Neural Architecture Search

arXiv:2606. 29582v1 Announce Type: cross Abstract: Bilevel optimization has become an influential and widely adopted framework for addressing hierarchical optimization problems in machine learning, providing an effective approach to modeling the interaction between two levels of optimization, with applications such as hyperparameter tuning, meta-learning, adversarial training, and data poisoning.

By Abhishek Shukla, Ankur Sinha, Faiz Hamid
arXiv AI
Jun 9

A Multi-Agent System for IPMSM Design Optimization via an FEA-AI Hybrid Approach

arXiv:2606. 09037v1 Announce Type: new Abstract: Interior permanent magnet synchronous motor (IPMSM) design requires balancing conflicting objectives and multi-physics constraints, while modern optimization workflows face three bottlenecks: manual problem setup, high finite element analysis (FEA) cost, and unreliable surrogate-based search in sparse or out-of-distribution regions.

By Jinseong Han, Sunwoong Yang, Namwoo Kang
arXiv Machine Learning
Jun 8

Amortized Neural Optimization for Pre-Layout Signal Integrity Design Space Exploration using Differentiable Surrogates

arXiv:2606. 07463v1 Announce Type: cross Abstract: Pre-layout design space exploration (DSE) for high-speed signal integrity (SI) analysis is often limited by the computational cost of simulations and iterative optimization algorithms within modern electronic design automation (EDA) workflows.

By Julian With\"oft, Werner John, Emre Ecik, Ralf Br\"uning, J\"urgen G\"otze
arXiv Machine Learning
Sep 24

tidyHEBO: Robust General-Purpose Bayesian Optimization with Model-Consistent Warping and Pareto Search

tidyHEBO is a BoTorch-native Bayesian optimization tool that jointly applies Yeo-Johnson output warping to a Gaussian‑process surrogate, evaluates acquisition functions on the original objective scale, and conducts constrained cumulative Pareto search across multiple acquisition criteria. Using only default settings, it outperformed other methods on the Olympus benchmark and performed strongly on synthetic, Needle‑in‑a‑Haystack, and Bayesmark tasks, while adaptive batching offered a trade‑off between parallelization and optimization quality. These results position tidyHEBO as a robust, reproducible optimizer suitable for diverse practical problems, including scientific applications and hyperparameter tuning.

By L. A. Zhukov, E. V. Shaburova, D. V. Antonets
arXiv Statistics ML
Sep 11

Learning-Based Surrogate Method for Stochastic Optimization under Decision-Dependent Uncertainty with Adaptive Random Designs

The paper introduces a learning-based surrogate approach for stochastic optimization problems where uncertainty depends on the decision, modeled via a nonparametric regression. It constructs a surrogate that embeds iteratively updated Jacobian estimates, using an adaptive random design that focuses sampling near the current iterate to achieve dimension‑independent convergence of the Jacobian estimates. The resulting learning‑based stochastic prox‑linear (L‑SPL) algorithm demonstrates nonasymptotic convergence rates and outperforms existing methods in sample efficiency and objective value in numerical experiments.

By Boyang Shen, Junyi Liu