arXiv Machine Learning

Satisficing Regret Minimization in Bandits: Constant Rate and Light-Tailed Distribution

The paper introduces SELECT, an algorithmic framework for satisficing regret minimization in bandit problems, achieving constant expected satisficing regret when a satisficing arm exists. A variant, SELECT‑LITE, further ensures a light‑tailed satisficing regret distribution while maintaining constant expected regret in the realizable case and sub‑linear standard regret otherwise. Experiments on synthetic data and a real‑world dynamic pricing scenario demonstrate the practical effectiveness of both algorithms.

arXiv AI

Sep 2

Bandits in Prod: Hyperparameter Optimization at Inference Time

The paper introduces Online Hyperparameter Optimization (OHPO), framing it as an infinitely many‑armed bandit problem over mixed and conditional search spaces. It proposes the IMABO framework, which couples any bandit policy with any oracle for proposing new configurations, and presents IMOSS—a restart‑free anytime policy with provable regret bounds. Experiments show that IMABO, combined with practical oracles such as TPE, an incumbent‑mutation oracle, and a pretrained tabular foundation model, outperforms random search across a range of settings from classical ML models to LLM‑based agents.

By Louis Abraham, Tuan-Anh Nguyen, Nicolas Devatine

llms agents reinforcement-learning

arXiv Machine Learning

Aug 3

Parameter-Free Heavy-Tailed Bandits

arXiv:2607. 29460v1 Announce Type: new Abstract: Heavy-tailed distributions arise naturally in sequential decision-making problems such as financial investment, online advertising, and network management, where rare but extreme outcomes can dominate performance.

By Gianmarco Genalti, Alberto Maria Metelli

arXiv Machine Learning

2d ago

Regret Analysis of Retry-Based Bandits

arXiv:2605.20854v3 Announce Type: replace Abstract: We provide the first regret analysis of ReMax in stochastic multi-armed bandits. Originally introduced for reinforcement learning, ReMax is motivat...

By Bingkui Tong, Junpei Komiyama, Soichiro Nishimori, Paavo Parmas

reinforcement-learning

arXiv Machine Learning

Jun 2

Finite-Time Regret Analysis of Retry-Aware Bandits

arXiv:2605. 20854v2 Announce Type: replace Abstract: We study a stochastic bandit algorithm motivated by retry-aware objectives that value the best outcome among multiple attempts, such as pass@$k$ and max@$k$.

By Bingkui Tong, Junpei Komiyama, Soichiro Nishimori, Paavo Parmas

reinforcement-learning

arXiv Machine Learning

Aug 4

Meritocratic Fairness via $K$-Shapley Values in Budgeted Combinatorial Bandits with Full-Bandit Feedback

arXiv:2605. 00762v2 Announce Type: replace Abstract: We study meritocratic fairness in budgeted combinatorial multi-armed bandits with full-bandit feedback, where a learner selects at most $K$ arms per time step and observes only the noisy aggregate reward of the selected set.

By Shradha Sharma, Shweta Jain, Swapnil Dhamal

reinforcement-learning safety

arXiv Machine Learning

Jun 2

Tree-Guided Identify-Then-Exploit: A Unified Framework of Best Arm Identification and Regret Minimization for Dueling Bandits

arXiv:2606. 01799v1 Announce Type: new Abstract: We study $N$-armed stochastic dueling bandits under the Condorcet-winner assumption, where three widely adopted objectives are considered: best-arm identification (BAI), weak regret, and strong regret.

By Pu Wang, Yao-Xiang Ding

arXiv Machine Learning

Jun 16

Best Arm Identification with Minimal Regret

arXiv:2409. 18909v2 Announce Type: replace Abstract: Motivated by real-world applications that necessitate responsible experimentation, we introduce the problem of best arm identification (BAI) with minimal regret.

By Junwen Yang, Vincent Y. F. Tan, Tianyuan Jin

agents reinforcement-learning

arXiv Statistics ML

2d ago

Multi-User mmWave Beam and Rate Adaptation via Combinatorial Satisficing Bandits

arXiv:2604.14908v2 Announce Type: replace-cross Abstract: We study downlink beam and rate adaptation in a multi-user mmWave MISO system where multiple base stations (BSs), each using analog beamformi...

By Emre \"Ozy{\i}ld{\i}r{\i}m, Bar{\i}\c{s} Yayc{\i}, Umut Eren Akturk, Cem Tekin

reinforcement-learning efficiency safety

arXiv Machine Learning

Jul 14

Oracle-Efficient Combinatorial Semi-Bandits

arXiv:2510. 21431v2 Announce Type: replace-cross Abstract: We study the combinatorial semi-bandit problem where an agent selects a subset of base arms and receives individual feedback.

By Jung-hun Kim, Milan Vojnovi\'c, Min-hwan Oh

agents reinforcement-learning

arXiv Machine Learning

Jun 29

Taming the Monster Every Context: Complexity Measure and Unified Framework for Offline-Oracle Efficient Contextual Bandits

arXiv:2602. 09456v2 Announce Type: replace Abstract: We propose an algorithmic framework, Offline Estimation to Decisions (OE2D), that efficiently reduces contextual bandit learning with general reward function approximation to offline regression.

By Hao Qin, Chicheng Zhang

reinforcement-learning

arXiv Machine Learning

Sep 22

Multi-Armed Bernoulli Bandits via Minimax Single-Arm Stopping

arXiv:2609. 22690v1 Announce Type: new Abstract: We develop an index policy for finite-horizon Bernoulli multi-armed bandits from minimax solutions to single-arm bandit (SAB) problems.

By Huikang Liu, Zhengchao Wang, Daniel Kuhn, Wolfram Wiesemann

reinforcement-learning benchmarks

arXiv Machine Learning

4d ago

Learning When to Update: A Near-Optimal Timing Bandit Approach

arXiv:2609.37932v1 Announce Type: new Abstract: Systems operating in dynamic environments require timely updates to sustain performance. For resource-intensive systems such as machine learning models...

By Qiulin Lin, Junyan Su, Liyuan Wang, Minghua Chen

reinforcement-learning

Related stories