The paper introduces Risk‑Sensitive Satisficing (RS), a bandit learning rule that evaluates actions based on their past frequency and their surplus or shortfall relative to an aspiration level. RS can be reformulated as Attention Competitive Bayes (ACB), a Bayesian‑like estimator where the aspiration level serves as a prior mean and the limited number of trials limits how strongly evidence can override this prior. ACB predicts that unchosen actions gradually revert toward the aspiration level (pseudo‑forgetting) and that when the aspiration is unattainable, exploration never fully settles, reducing regret in simulations.
By Moto Shinriki, Yu Kono, Tatsuji Takahashi
arXiv:2610.19046v1 Announce Type: new
Abstract: Information-theoretic exploration is of primary interest in methodological studies of reinforcement learning (RL) and computational cognitive neuroscie...
By Melih Kandemir, Mustafa Mert \c{C}elikok
The paper introduces a two-player zero-sum repeated game between a learner and nature that simultaneously captures Bayesian updating and an exact decomposition of exponential-weights regret. The game’s terminal payoff reflects the maximum gain a comparator can achieve given a fixed relative entropy from the prior, while the one-step constraint limits nature’s move by an information budget. The resulting regret splits into three precise components—per-round information loss, an additive retempering drift, and the comparator’s information relative to the prior—providing a unified framework that explains concentration phenomena, large-deviation bounds, and various learning methods such as bandits, posterior sampling, aggregation, and boosting.
By Akshay Balsubramani
arXiv:2605. 20854v2 Announce Type: replace Abstract: We study a stochastic bandit algorithm motivated by retry-aware objectives that value the best outcome among multiple attempts, such as pass@$k$ and max@$k$.
By Bingkui Tong, Junpei Komiyama, Soichiro Nishimori, Paavo Parmas
arXiv:2609. 15248v2 Announce Type: replace-cross Abstract: In bandits with probing, a learner selects at most $k$ of $n$ arms per round, earns the largest of their rewards, and competes with the best fixed single arm.
By Yongjie Guan
arXiv:2605.20854v3 Announce Type: replace
Abstract: We provide the first regret analysis of ReMax in stochastic multi-armed bandits. Originally introduced for reinforcement learning, ReMax is motivat...
By Bingkui Tong, Junpei Komiyama, Soichiro Nishimori, Paavo Parmas
The paper introduces E‑Process‑Authorized Thompson Sampling (e‑ATS), a bandit algorithm that uses full‑history and discounted Beta states to decide when to forget past rewards in non‑stationary environments. An anytime‑valid e‑process first authorizes the discounted state, after which a reversible relevance score determines its influence; before authorization, e‑ATS behaves exactly like optimistic Thompson sampling (OTS). Experiments show that removing the authorization step increases mean normalized dynamic pseudo‑regret by 38.4 % on a registered suite but decreases it by 7.5 % on a literature‑derived replay suite, indicating that evidence controls when adaptation begins rather than guaranteeing improvement.
By Mayand Gulati, Kerong Wang, WeiChen Au
arXiv:2606. 00913v1 Announce Type: cross Abstract: Multi-arm bandit algorithms are increasingly used in online platforms, clinical trials, and social science experiments, but valid statistical inference on their performance remains an open challenge.
By Samya Praharaj, Chih-Yu Chang, Koulik Khamaru, Kelly W. Zhang
The paper introduces Persistent Candidate LinUCB (PC‑LinUCB), a variant of the LinUCB algorithm for linear bandits that reduces per‑round score evaluations from ≥×K to ≥×(k+q) by maintaining a small set of active actions and sampling challengers from the remaining actions. It provides high‑probability regret bounds that include a standard learning term and an additive discovery cost of order ≥×((K-k)×log(K-k))/q, which does not grow with the horizon. Experiments demonstrate that PC‑LinUCB accelerates LinUCB while keeping regret comparable, and the persistent candidate approach can be applied beyond LinUCB.
By Bingkui Tong, Daria Korovaitceva, Marina Sheshukova, Sergey Samsonov, Junpei Komiyama
The paper studies the trade‑off between maximizing rewards and accurately estimating current treatment effects in nonstationary multi‑armed bandits. For $K$‑armed bandits with H"older‑smooth mean rewards, the authors derive minimax bounds on expected dynamic regret under a high‑probability simultaneous coverage requirement for all current effects. They propose the TRACE algorithm, which achieves a regret–precision trade‑off that separates reward‑learning regret from the unavoidable cost of refreshing current‑effect estimates, and extend the method with an empirical‑Bernstein e‑process to adapt exploration while preserving worst‑case guarantees.
By Sharan Sahu, Martin T. Wells
The paper introduces Online Hyperparameter Optimization (OHPO), framing it as an infinitely many‑armed bandit problem over mixed and conditional search spaces. It proposes the IMABO framework, which couples any bandit policy with any oracle for proposing new configurations, and presents IMOSS—a restart‑free anytime policy with provable regret bounds. Experiments show that IMABO, combined with practical oracles such as TPE, an incumbent‑mutation oracle, and a pretrained tabular foundation model, outperforms random search across a range of settings from classical ML models to LLM‑based agents.
By Louis Abraham, Tuan-Anh Nguyen, Nicolas Devatine
arXiv:2611. 05580v1 Announce Type: cross Abstract: We study repeated general games over $T$ rounds between a utility-maximizing optimizer and a learning agent, where budget constraints couple the players' decisions across rounds.
By Giannis Fikioris, Balasubramanian Sivan, \'Eva Tardos