arXiv AI

Reserve-Aware Contrast Certificates for Conservative Bandits with Uncertain Baselines

The paper introduces Reserve-C4B, a method for conservative bandits that ensures improvement over an incumbent policy while respecting a performance budget, even when the incumbent’s reward is uncertain. By focusing on the baseline-relative contrast and using a shared confidence set, the approach derives an exact expression for the avoidable penalty and a tighter admissibility test at each history. The method incorporates a reserve ledger to separate statistical evidence from performance deficit and a prefix-refresh extension to recertify decisions without discarding prior credit, achieving high-probability conditional-mean performance guarantees for linear rewards.

arXiv Machine Learning

Sep 18

Odds-Ratio Thompson Sampling: A Specification and Design Guide for Contrast-Based Multi-Armed Bandits

The paper introduces Odds‑Ratio Thompson Sampling (OR‑TS), a method for batched multi‑armed bandits that updates the joint posterior over log‑odds contrasts and refits the shared level in each batch, rather than carrying over absolute reward rates. It presents a Bayesian bandit agent with controls for decay of past evidence and aggressiveness of allocation, and evaluates OR‑TS against traditional absolute‑rate memory across 86 public A/B series and synthetic environments. Results show that when the shared level varies significantly, OR‑TS outperforms absolute‑rate memory, reducing regret and ensuring the best arm receives more traffic, while also handling cases where contrasts themselves shift.

By Sulgi Kim

agents reinforcement-learning

arXiv Machine Learning

Jun 24

Flow-Corrected Thompson Sampling for Non-Stationary Contextual Bandits

arXiv:2606. 23933v1 Announce Type: cross Abstract: We study non-stationary linear contextual bandits where the reward model drifts over time, rendering classical contextual bandit algorithms brittle because historical data becomes systematically biased.

By AmirHossein Naghdi, Ali Baheri

reinforcement-learning benchmarks

arXiv Machine Learning

Aug 4

Conformal bandits: bringing statistical validity and reward efficiency under weak arm separability

arXiv:2512. 09850v2 Announce Type: replace Abstract: We introduce Conformal Bandits, a novel framework integrating Conformal Prediction (CP) into bandit problems, a classic paradigm for sequential decision-making under uncertainty.

By Simone Cuonzo, Nina Deliu

reinforcement-learning

arXiv Machine Learning

Jun 2

Adaptive Exploration for Latent-State Bandits

arXiv:2602. 05139v3 Announce Type: replace Abstract: We study bandits whose rewards depend on an unobserved Markov state that evolves independently of the learner's actions.

By Jikai Jin, Kenneth Hung, Sanath Kumar Krishnamurthy, Baoyi Shi, Congshan Zhang

reinforcement-learning safety

arXiv Machine Learning

Jun 2

Bandit Simulation for Average Reward Inference

arXiv:2606. 00913v1 Announce Type: cross Abstract: Multi-arm bandit algorithms are increasingly used in online platforms, clinical trials, and social science experiments, but valid statistical inference on their performance remains an open challenge.

By Samya Praharaj, Chih-Yu Chang, Koulik Khamaru, Kelly W. Zhang

reinforcement-learning

arXiv Machine Learning

Aug 12

Weighted Sequential Bayesian Inference for Non-Stationary Linear Contextual Bandits

arXiv:2307. 03587v4 Announce Type: replace Abstract: In non-stationary linear contextual bandits, existing efficient algorithms typically rely on the Weighted Regularized Least-Squares (WRLS) estimator.

By Nicklas Werge, Yi-Shan Wu, Abdullah Akg\"ul, Melih Kandemir

benchmarks

arXiv AI

Jun 17

Optimism Stabilizes Thompson Sampling for Adaptive Inference

arXiv:2602. 06014v2 Announce Type: replace-cross Abstract: Thompson sampling (TS) is widely used for stochastic multi-armed bandits, yet its inferential properties under adaptive data collection are subtle.

By Shunxing Yan, Han Zhong

arXiv Machine Learning

Jun 2

Finite-Time Regret Analysis of Retry-Aware Bandits

arXiv:2605. 20854v2 Announce Type: replace Abstract: We study a stochastic bandit algorithm motivated by retry-aware objectives that value the best outcome among multiple attempts, such as pass@$k$ and max@$k$.

By Bingkui Tong, Junpei Komiyama, Soichiro Nishimori, Paavo Parmas

reinforcement-learning

arXiv Machine Learning

Aug 10

Bootstrap-Conditioned Action Selection with Tabular Foundation Models

arXiv:2608. 06559v1 Announce Type: new Abstract: Contextual bandits offer a natural framework for sample-efficient personalization, but practical deployment remains difficult under sparse, biased interaction data, unreliable uncertainty estimates, and severe cold starts.

By Devansh Gupta, Shiv Tavker, Dmitry Efimov, Suchitra Sathyanarayana, Gitanjali Bhutani, Boris N. Oreshkin

llms reinforcement-learning

arXiv Machine Learning

Sep 23

Sharp Characterization of Bias in Post-Bandit Inference

arXiv:2608.01069v2 Announce Type: replace Abstract: Bandit algorithms generate data for downstream inference, but adaptive sampling biases post-bandit sample means. We analyze this bias for stable in...

By Lisu Wang, Yilun Chen, Jiaqi Lu

reinforcement-learning safety

arXiv Machine Learning

Jun 30

Randomized Exploration for Linear Bandits via Absolute Perturbations

arXiv:2606. 28616v1 Announce Type: new Abstract: In stochastic linear bandits, the canonical Upper Confidence Bound (UCB) algorithm admits a simple frequentist regret analysis but can be computationally demanding, while Thompson Sampling (TS) is computationally attractive yet typically harder to analyze due to its non-optimistic nature.

By Toshinori Kitamura, Shuai Liu, Csaba Szepesv\'ari

arXiv Machine Learning

2d ago

Rising Multi-Armed Bandits with Known Horizons

arXiv:2602.10727v3 Announce Type: replace Abstract: Rising Multi-Armed Bandits (RMABs) model sequential decision problems where each arm's expected reward improves with repeated pulls. In such proble...

By Seockbean Song, Chenyu Gan, Youngsik Yoon, Siwei Wang, Wei Chen, Jungseul Ok

reinforcement-learning benchmarks

Related stories