arXiv Machine Learning

Bilevel Optimization of Topology and Hyperparameters (BOTH)

The paper introduces BOTH, a method that differentiates topology optimization (TO) itself to compute hypergradients for tuning hyperparameters alongside the primary design optimization. By evaluating only one or two TO steps, the approach provides sufficient information and scales to thousands of hyperparameters with a cost comparable to a few standard TO runs. Experiments on stress‑constrained and compliance problems, including a neural‑parameterized density field, demonstrate the effectiveness of this joint optimization strategy.

arXiv AI

Sep 24

KATOsuper: Surrogate-accelerated neural topology optimization with sensitivity-consistent Fourier neural operators

KATOsuper is an objective‑agnostic framework that accelerates neural topology optimization by coupling neural‑reparameterized TO with a Sensitivity‑Consistent Fourier Neural Operator (SC‑FNO). It uses a forward_split architecture to ensure that sensitivities derived via automatic differentiation remain consistent with predicted objectives, enabling stable optimization. The method demonstrates significant deployment‑time speedups (15–110×) over MATLAB baselines while preserving optimality across 2D and 3D benchmark problems, including compliance and stress minimization, and supports zero‑shot extrapolation to higher resolutions.

By Shengyu Yan, Jasmin Jelovica

rag benchmarks

arXiv AI

Aug 26

Constrained Hyperparameter Optimization for Streaming Data

arXiv:2608.24712v1 Announce Type: cross Abstract: Optimization of hyperparameters is a critical factor to obtain optimal model performance. While existing research has predominantly concentrated on b...

By Bruno Veloso, Jo\~ao Gama

arXiv AI

Jun 30

Bilevel Optimization for Neural Architecture Search

arXiv:2606. 29582v1 Announce Type: cross Abstract: Bilevel optimization has become an influential and widely adopted framework for addressing hierarchical optimization problems in machine learning, providing an effective approach to modeling the interaction between two levels of optimization, with applications such as hyperparameter tuning, meta-learning, adversarial training, and data poisoning.

By Abhishek Shukla, Ankur Sinha, Faiz Hamid

fine-tuning safety

arXiv AI

Jun 2

On the Generalization in Topology Optimization via Sensitivity-Conditioned Bernoulli Flow Matching

arXiv:2606. 02179v1 Announce Type: cross Abstract: Surrogate models for topology optimization (TO) exhibit highly variable out-of-distribution (OOD) generalization under distribution shifts such as changing loads or boundary conditions, yet the source of this variability remains unclear.

By Mohammad Rashed, Duarte F. Valoroso Madeira, Babak Gholami, Caglar Guerbuez, Yunjia Yang, Nils Thuerey

diffusion benchmarks

Hugging Face Trending Papers

Aug 23

KPI-Conditioned Generative Design of Automotive Hood Inner Panels: A Two-Stage Retrieval-Generation Pipeline with Surrogate-Based Performance Estimation

An inner hood panel must meet a deflection target, stay below a stress limit, and hit a mass target. Machine-learned surrogates have made the forward direction, geometry to performance, fast and routi...

arXiv AI

Jun 18

Surrogate Benchmarks for Model Merging Optimization

arXiv:2509. 02555v2 Announce Type: replace-cross Abstract: Model merging techniques aim to integrate the abilities of multiple models into a single model.

By Rio Akizuki, Yuya Kudo, Nozomu Yoshinari, Yoichi Hirose, Toshiyuki Nishimoto, Kento Uchida, Shinichi Shirakawa

llms benchmarks

arXiv Machine Learning

Jul 31

Towards Stability of Parameter-Free Optimization

arXiv:2405. 04376v4 Announce Type: replace Abstract: Hyperparameter tuning, particularly the selection of an appropriate learning rate in adaptive gradient training methods, remains a challenge.

By Yijiang Pang, Shuyang Yu, Bao Hoang, Jiayu Zhou

arXiv Machine Learning

Aug 13

Trust Region Constrained Bayesian Optimization with Penalized Constraint Handling

arXiv:2603. 24567v2 Announce Type: replace-cross Abstract: Constrained optimization in high-dimensional black-box settings is difficult due to expensive evaluations, the lack of gradient information, and complex feasibility regions.

By Raju Chowdhury, Tanmay Sen, Biswabrata Pradhan

benchmarks

arXiv Machine Learning

Sep 17

Benchmarking Tabular Foundation Models as Surrogates in Expensive Evolutionary Optimization

The paper evaluates the Tabular Prior-data Fitted Network (TabPFN) as a surrogate model in surrogate‑assisted evolutionary algorithms (SAEAs) for expensive optimization problems. Through extensive experiments in both offline and online settings across a range of problem types—including single‑objective, multi‑objective, constrained, combinatorial, mixed‑variable, and engineering tasks—the study finds that TabPFN’s effectiveness varies strongly with the problem characteristics. The authors conclude that TabPFN should be used selectively, with customized model management and algorithm design tailored to data availability, landscape complexity, and search‑space properties.

By Lu Han, Jin Wang, Yuchen Li, Haoran Gu, Shulei Liu, Ziyang Shi, Wenao Lu, Handing Wang

benchmarks

arXiv Machine Learning

Aug 3

Dynamic Priors in Bayesian Optimization for Hyperparameter Optimization

arXiv:2511. 02570v3 Announce Type: replace Abstract: Bayesian optimization (BO) is a widely used approach to hyperparameter optimization (HPO).

By Lukas Fehring, Marcel Wever, Maximilian Splieth\"over, Leona Hennig, Henning Wachsmuth, Marius Lindauer

benchmarks

arXiv Machine Learning

Jun 19

Evolutionary Two-Stage Hyperparameter Optimization Strategies for Physics-Informed Neural Networks

arXiv:2606. 20442v1 Announce Type: new Abstract: Physics-Informed Neural Networks (PINNs) solve Partial Differential Equations (PDEs) by embedding physical laws into neural network training.

By Fedor Buzaev (HSE University), Dmitry Efremenko (HSE University), Egor Bugaev (HSE University), Andrei Ermakov (HSE University, AXXX), Denis Derkach (HSE University), Daria Pugacheva (HSE University, AXXX), Fedor Ratnikov (HSE University)

rag

arXiv AI

Jul 7

OmniOpt: Taxonomy, Geometry, and Benchmarking of Modern Optimizers

arXiv:2607. 04033v1 Announce Type: cross Abstract: Optimizer selection for large-scale model training has become a system-level design decision constrained jointly by compute, memory, tuning budget, and task diversity, yet the landscape of over one hundred methods remains fragmented.

By Siyuan Li, Jiabao Pan, Yumou Liu, Zhuoli Ouyang, Xin Jin, Xinglong Xu, Jingxuan Wei, Shengye Pang, Jintao Che, Xuanhe Zhou, Conghui He, Cheng Tan

llms computer-vision benchmarks

Related stories