← Back to all news
arXiv Machine Learning

Meta-Frieren: Beyond Single Level Online Learning

The paper introduces Meta-Frieren, a framework for two‑stage online optimization where a meta‑learner selects an agent‑environment pair before the agent interacts and receives a reward. It derives a UCB‑style criterion for meta‑level selection, extends prior guarantees, and instantiates the framework with GP‑UCB under Bayesian and frequentist settings. Additionally, it presents the first high‑probability regret bound for Feel‑Good Thompson Sampling, applies it to finite and parametric function classes, and demonstrates Meta‑Frieren’s effectiveness in hyperparameter optimization across multiple model classes through experiments.

Related stories

arXiv Machine Learning
Sep 16

Meta-LinEXP3: Online-within-Online Learning for Adversarial Linear Contextual Bandits

Meta-LinEXP3 is an online-within-online algorithm designed for adversarial linear contextual bandits with random action sets. It builds a task-level prior from completed tasks to guide an inner LinEXP3 learner, achieving an σO(√n) per‑task regret when context distributions are known and an σO(n^{2/3}) regret with a past‑only regularized moment estimator when they are unknown. The paper also links prior accuracy to transfer regret, showing that better priors yield sublinear, transfer‑dependent regret across tasks, and demonstrates the method on structured hyperspectral tensor sampling.

By Hao Li, Jie Xu, Zheng Xie
reinforcement-learningsafety
More like this →
arXiv Machine Learning
Oct 19

Empirical Bayesian Multi-Bandit Learning

The paper introduces a hierarchical Bayesian framework for multi-task contextual bandits, capturing both heterogeneity and correlations among bandit instances. It proposes an empirical Bayesian method to estimate the covariance matrix of the prior, leading to two new algorithms—ebmTS and ebmUCB—that incorporate this prior into decision making. The authors provide frequentist regret bounds and show through synthetic and real-world experiments that these algorithms outperform existing methods, especially in complex settings.

By Xia Jiang, Rong J. B. Zhu
reinforcement-learning
More like this →
arXiv Machine Learning
Aug 31

Meta-Prompt Optimization for LLM-Based Sequential Decision Making

The paper introduces EXPO, an algorithm that automatically optimizes the meta-prompt—specifically the task description and meta-instruction—for large language model agents in sequential decision-making tasks such as Bayesian optimization and multi-armed bandits. Building on adversarial bandit techniques to handle non-stationary rewards, the authors extend EXPO to EXPO-ES, which also optimizes exemplars (historical interactions) within the meta-prompt. Experiments demonstrate that these methods significantly improve the performance of LLM-based agents in sequential decision-making scenarios.

By Mingze Kong, Zhiyong Wang, Yao Shu, Zhongxiang Dai
llmsagentsreinforcement-learningsafety
More like this →
arXiv Machine Learning
Oct 15

High-Probability Regret Bounds for Feel-Good Thompson Sampling

The paper presents the first high‑probability regret bounds for both cumulative and current‑step variants of Feel‑Good Thompson Sampling (FGTS) in contextual bandits with general realizable function approximation. The bounds rely on decoupling coefficients and prior mass near the true reward function, and in the linear setting they achieve ≠d√T log(2T)+√T log(1/δ) regret with probability at least 1‑δ. This matches the dimension and horizon dependence of upper‑confidence‑bound algorithms up to logarithmic factors, while the high‑probability cost adds only linear dependence on log(1/δ).

By Xuheng Li, Qingyue Zhao, Quanquan Gu
llms
More like this →
arXiv AI
Sep 2

Bandits in Prod: Hyperparameter Optimization at Inference Time

The paper introduces Online Hyperparameter Optimization (OHPO), framing it as an infinitely many‑armed bandit problem over mixed and conditional search spaces. It proposes the IMABO framework, which couples any bandit policy with any oracle for proposing new configurations, and presents IMOSS—a restart‑free anytime policy with provable regret bounds. Experiments show that IMABO, combined with practical oracles such as TPE, an incumbent‑mutation oracle, and a pretrained tabular foundation model, outperforms random search across a range of settings from classical ML models to LLM‑based agents.

By Louis Abraham, Tuan-Anh Nguyen, Nicolas Devatine
llmsagentsreinforcement-learning
More like this →