arXiv AI By Yiming Yao, Fei Liu, Liang Zhao, Xi Lin, Yilu Liu, Qingfu Zhang

FoMEMO: Towards Foundation Models for Expensive Multi-objective Optimization

Read the original on arXiv AI →

FoMEMO is a foundation model designed for expensive multi-objective optimization. It transforms the problem into preference-wise aggregated posterior learning tasks, allowing scalable pre-training on vast synthetic datasets without extensive real-world experiments. At inference, the model uses observed trajectories and user preferences to perform in-context posterior prediction and optimize acquisition functions, achieving strong generalization without further training.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Oct 28

BoLT: A Benchmark to Democratize Black-box Optimization Research for Expensive LLM Tasks

BoLT is a new benchmark designed to make black-box optimization research for large language model (LLM) tasks more accessible. It presents a suite of realistic LLM optimization problems—covering multi‑fidelity, multi‑objective, heteroscedastic noise, and high‑dimensional search spaces—grounded in thousands of real experiments and represented by lightweight surrogate models. The authors evaluate a wide range of Bayesian and other black‑box optimization methods on BoLT, finding that some BO techniques consistently outperform others and revealing gaps in current methods for LLM optimization.

By Ruth Wan Theng Chew, Zhiliang Chen, Apivich Hemachandra, Bryan Kian Hsiang Low
arXiv Statistics ML
Nov 9

Simple Extensions of Single-Objective Acquisition Functions and Hedge Strategies for Multi-Objective Bayesian Optimization

The paper proposes a simple framework that extends standard single-objective Bayesian optimization acquisition functions to the multi-objective setting using a hypervolume-based approach, and adapts hedge strategies for multi-objective use. This method requires minimal changes to existing pipelines and preserves the intuitive interpretation and computational efficiency of the original strategies. Experiments on synthetic and real-world benchmarks show that these extensions consistently match or outperform more complex state-of-the-art multi-objective Bayesian optimization methods.

By Haris Moazam Sheikh