arXiv Machine Learning
Jun 10
mlr3mbo: Bayesian Optimization in R
arXiv:2603. 29730v2 Announce Type: replace-cross Abstract: We present mlr3mbo, a modular toolbox for Bayesian optimization in R.
Scalable Batch Bayesian Optimization Via Subspace Acquisition Functions
arXiv:2411. 16206v3 Announce Type: replace-cross Abstract: Extending Bayesian optimization to batch evaluation can enable the designer to make the most use of parallel computing technology.
Related stories
arXiv Machine Learning
Sep 17
Gradient Descent with Stochastic Subspaces via Persistence of Memory
The paper introduces a novel technique called "persistence of memory" to enhance stochastic subspace methods for large‑scale optimisation. By using a weakly correlated guidance vector that is refreshed only at wide intervals, the method provides a structured direction for random subspace descent. The authors demonstrate that this guidance can be efficiently computed in sparse or minibatch settings and present the first theoretical analysis of classical SSD methods for sparse functions, showing alignment with low‑lying Hessian eigenvectors near the optimum.
arXiv Machine Learning
Aug 13
Trust Region Constrained Bayesian Optimization with Penalized Constraint Handling
arXiv:2603. 24567v2 Announce Type: replace-cross Abstract: Constrained optimization in high-dimensional black-box settings is difficult due to expensive evaluations, the lack of gradient information, and complex feasibility regions.
arXiv Machine Learning
Jun 30
B3O: Scalable Boltzmann Batch Bayesian Optimization
arXiv:2606. 30228v1 Announce Type: new Abstract: Modern engineering workflows increasingly rely on massive parallel simulation, driving the need for scalable, large-batch Bayesian Optimization (BO).
arXiv Machine Learning
Jun 16
Concrete Subspace Learning based Interference Elimination for Multi-task Model Fusion
arXiv:2312. 06173v2 Announce Type: replace Abstract: Merging models fine-tuned from a common, extensively pre-trained large model but specialized for different tasks has been demonstrated as a cheap and scalable strategy to construct a multi-task model that performs well across diverse tasks.
arXiv Machine Learning
Jun 30
Warm-Starting Iterative Gaussian Processes for Faster Sequential Inference
arXiv:2511. 16340v2 Announce Type: replace Abstract: Efficient Gaussian process (GP) inference is critical for sequential decision-making tasks such as active learning, online prediction, and Bayesian optimization.
arXiv Machine Learning
Jul 7
Local Constrained Bayesian Optimization
arXiv:2603. 07965v2 Announce Type: replace-cross Abstract: Bayesian optimization (BO) for high-dimensional constrained problems remains a significant challenge due to the curse of dimensionality.
arXiv Machine Learning
Sep 25
MF-SCBO : Multi-fidelity Scalable Constrained Bayesian Optimization
MF-SCBO is a new multi‑fidelity extension of Scalable Constrained Bayesian Optimization designed for high‑dimensional black‑box functions with black‑box constraints. It handles an arbitrary number of fidelity levels and non‑nested sampling, addressing gaps in existing methods. Experiments on standard benchmarks and challenging problems show that MF‑SCBO generally converges faster than both single‑fidelity SCBO and other multi‑fidelity approaches in high‑dimensional constrained settings.
arXiv Machine Learning
Aug 27
Gradient-based Sample Selection for Faster Bayesian Optimization
The paper introduces Gradient-based Sample Selection Bayesian Optimization (GSSBO), a method that builds the Gaussian process surrogate on a strategically chosen subset of samples rather than the full dataset. By using gradient information to eliminate redundant points while keeping diversity and representativeness, GSSBO achieves sublinear regret bounds and reduces the cubic computational cost of standard BO. Experiments on synthetic and real-world tasks show that this approach maintains comparable optimization performance while significantly cutting GP fitting time and resource usage.
arXiv AI
Sep 18
Compressed Active Subspaces for Scalable Bayesian Inference
The paper introduces Compressed Active Subspaces (CAS), a scalable method for Bayesian inference in high‑dimensional models. CAS first compresses model parameters via a structured isometric embedding, then constructs the active subspace in this reduced space, dramatically lowering memory requirements. Experiments on neural networks of growing size show that CAS preserves predictive performance and provides robust uncertainty estimates while enabling inference where traditional active subspace methods fail.
arXiv Machine Learning
Sep 14
Nonlinear Dimensionality Reduction Techniques for Bayesian Optimization
The paper investigates nonlinear dimensionality reduction for Bayesian optimisation (BO) by transforming high‑dimensional black‑box optimisation problems into a sequence of low‑dimensional latent‑space BO (LSBO) tasks. It extends earlier linear embedding approaches by using variational autoencoders (VAEs), deep metric loss, and adaptive retraining to better capture nonlinear structure, and couples LSBO with sequential domain reduction (SDR‑LSBO) to progressively narrow search domains. Experiments on GPU‑accelerated BoTorch with Matérn‑5/2 Gaussian‑process surrogates show that VAE‑based LSBO outperforms adaptive linear embeddings, and the authors provide a theoretical analysis of latent‑space error versus representation gap under PAC‑Bayes conditions.
arXiv Machine Learning
Sep 18
Bayesian Optimization with Rich Auxiliary Information via LLMs
The paper introduces Bayesian Optimization (BO) techniques that incorporate rich auxiliary information—such as training curves, expert notes, images, and prior knowledge—using large language models (LLMs). Three new methods are proposed to integrate this auxiliary data into BO, and they are evaluated on hyperparameter optimization benchmarks and a real-world nuclear fusion task. The results show that these LLM-enhanced BO methods consistently outperform standard BO and existing LLM-based optimization approaches.