Regime-Conditioned Evaluation in Multi-Context Bayesian Optimization
Read the original on arXiv Machine Learning →The paper introduces Regime-Conditioned Evaluation in Multi-Context Bayesian Optimization, arguing that average treatment effect estimates of acquisition strategies overlook critical conditional effects tied to prior quality, budget ratio, and metric. An audit of 40 transfer-BO studies shows that 98% never vary the budget-to-algorithm ratio, leading to unstable rankings—e.g., Greedy outperforms UCB at a low budget but not at a higher one. The authors propose the Portable Regime Score (PRS) to capture this transition, demonstrate its predictive power across 79 diverse conditions, and present RegimePlanner, an online method that adapts acquisition based on PRS and achieves superior performance in HPO-B and GDSC2 benchmarks. whyItMatters":"The study highlights that unconditional rankings of Bayesian optimization methods are unreliable, emphasizing the need to report regime-specific metrics like B/|A| and prior rank correlation to accurately assess acquisition performance."
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.