The paper introduces Regime-Conditioned Evaluation in Multi-Context Bayesian Optimization, arguing that average treatment effect estimates of acquisition strategies overlook critical conditional effects tied to prior quality, budget ratio, and metric. An audit of 40 transfer-BO studies shows that 98% never vary the budget-to-algorithm ratio, leading to unstable rankings—e.g., Greedy outperforms UCB at a low budget but not at a higher one. The authors propose the Portable Regime Score (PRS) to capture this transition, demonstrate its predictive power across 79 diverse conditions, and present RegimePlanner, an online method that adapts acquisition based on PRS and achieves superior performance in HPO-B and GDSC2 benchmarks. whyItMatters":"The study highlights that unconditional rankings of Bayesian optimization methods are unreliable, emphasizing the need to report regime-specific metrics like B/|A| and prior rank correlation to accurately assess acquisition performance."
arXiv:2610. 00445v1 Announce Type: new Abstract: Pairwise guide--transcript scores do not enforce conservation of a finite guide-loaded RISC pool when they are interpreted independently as occupancies.
StrictPRMBench is a replayable evaluation benchmark designed to audit fixed‑score PRM aggregation rules. It standardizes mathematical reasoning pools, scorer evidence, and deletion interventions so that different reducers confront identical selection problems. Across seven checkpoints and two benchmarks, the benchmark reveals that product aggregation yields higher deletion‑induced selection instability than geometric mean, with statistically significant differences in most cells.
arXiv:2608. 07859v1 Announce Type: new Abstract: Preferential Bayesian optimization (PBO) optimizes objectives accessible only through pairwise user comparisons.
arXiv:2608. 14551v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used for title-abstract screening in systematic reviews, but their decisions lack calibrated uncertainty.
arXiv:2606. 21641v2 Announce Type: replace-cross Abstract: Large language models (LLMs) have been proposed as hyperparameter-optimization (HPO) advisors that "warm-start" search from prior knowledge, proposing strong configurations in very few evaluations.
arXiv:2608. 00915v1 Announce Type: new Abstract: Uplift modeling (conditional-average-treatment-effect estimation) drives personalized targeting, yet published uplift benchmarks frequently disagree on which estimator performs best; we show the disagreement is substantially about metrics, not models.
The paper shows that Worst‑Case Optimal Recovery (OR) and Bayesian learning solve the same Gaussian‑quadratic‑Hilbert problems, linking the radius of information to a nugget‑optimized Gaussian process posterior variance. It evaluates three Bayesian systems, demonstrating that OR can outperform Bayesian methods in certain calibration and reproducibility metrics, yet split‑conformal and other approaches can beat OR in interval scoring, especially under covariate shift. The authors propose matching the guarantee tool to the data regime and auditing that regime first.
arXiv:2606. 17005v1 Announce Type: new Abstract: Public AI evaluations are often read as terminal leaderboards, yet the underlying evidence is a selective time series shaped by reporting rules, benchmark revisions, and missingness.
arXiv:2607. 18163v1 Announce Type: cross Abstract: PPO and the GRPO baseline studied here use clipped surrogate objectives whose favorable-direction saturation introduces an abrupt change in the scalar objective's derivative.
arXiv:2609.39634v1 Announce Type: cross Abstract: Common policy improvement methods, including TRPO, PPO, and GRPO, estimate policy improvement under the behavioral policy's state-visitation distribu...
arXiv:2609.37858v1 Announce Type: new Abstract: Many localized large language model (LLM) unlearning methods select a small parameter subset from a localization signal and keep it fixed during optimi...
arXiv:2607. 02104v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly used as cheap, scalable judges that compare candidate outputs pairwise.