The paper introduces Odds‑Ratio Thompson Sampling (OR‑TS), a method for batched multi‑armed bandits that updates the joint posterior over log‑odds contrasts and refits the shared level in each batch, rather than carrying over absolute reward rates. It presents a Bayesian bandit agent with controls for decay of past evidence and aggressiveness of allocation, and evaluates OR‑TS against traditional absolute‑rate memory across 86 public A/B series and synthetic environments. Results show that when the shared level varies significantly, OR‑TS outperforms absolute‑rate memory, reducing regret and ensuring the best arm receives more traffic, while also handling cases where contrasts themselves shift.
arXiv:2606. 23933v1 Announce Type: cross Abstract: We study non-stationary linear contextual bandits where the reward model drifts over time, rendering classical contextual bandit algorithms brittle because historical data becomes systematically biased.
arXiv:2607. 12389v1 Announce Type: cross Abstract: We consider Bayesian bandit models and prove that Thompson sampling makes at most twice the expected number of mistakes (selections of a suboptimal arm) as any other policy.
arXiv:2605. 20854v2 Announce Type: replace Abstract: We study a stochastic bandit algorithm motivated by retry-aware objectives that value the best outcome among multiple attempts, such as pass@$k$ and max@$k$.
arXiv:2602. 05139v3 Announce Type: replace Abstract: We study bandits whose rewards depend on an unobserved Markov state that evolves independently of the learner's actions.
arXiv:2608.01069v2 Announce Type: replace Abstract: Bandit algorithms generate data for downstream inference, but adaptive sampling biases post-bandit sample means. We analyze this bias for stable in...
arXiv:2602. 06014v2 Announce Type: replace-cross Abstract: Thompson sampling (TS) is widely used for stochastic multi-armed bandits, yet its inferential properties under adaptive data collection are subtle.
arXiv:2605.20854v3 Announce Type: replace Abstract: We provide the first regret analysis of ReMax in stochastic multi-armed bandits. Originally introduced for reinforcement learning, ReMax is motivat...
The paper introduces Reserve-C4B, a method for conservative bandits that ensures improvement over an incumbent policy while respecting a performance budget, even when the incumbent’s reward is uncertain. By focusing on the baseline-relative contrast and using a shared confidence set, the approach derives an exact expression for the avoidable penalty and a tighter admissibility test at each history. The method incorporates a reserve ledger to separate statistical evidence from performance deficit and a prefix-refresh extension to recertify decisions without discarding prior credit, achieving high-probability conditional-mean performance guarantees for linear rewards.
The paper introduces Decision‑Relevant Fresh Comparison (DRFC), a method for decentralized bandit systems with heterogeneous agents whose local reward changes may not affect the global best action. DRFC gathers balanced samples from all agents and only switches the common best arm when fresh global evidence indicates a change, yielding a dynamic regret bound that does not depend on the number of local changes. An anytime‑valid sliding‑window extension further handles gradual drift, and experiments on synthetic, semi‑real, and MovieLens‑1M data demonstrate that DRFC ignores decision‑irrelevant local changes while the extension avoids false switches.
arXiv:2609. 01761v1 Announce Type: cross Abstract: A system often has to act long before it learns whether the act worked: a recommender sees a click in seconds and a purchase in days.
The paper investigates the problem of sharing a single critic across multiple parallel environments in reinforcement learning. It shows that when environments assign different expected returns to the same state, a shared critic must reconcile conflicting value targets, which can distort advantage estimates and misguide policy updates. The authors propose a simple fix—providing the critic with the environment index—demonstrating through bandit models and experiments on CartPole, MuJoCo, BipedalWalker, and 16 Procgen games that this conditional critic stabilizes learning and boosts returns, achieving a 40.8% improvement in aggregate normalized return on unseen levels.
arXiv:2606. 09802v1 Announce Type: cross Abstract: We consider a variant of the linear contextual stochastic multi-armed bandits, where the learner must provide recommendations to a group of users, each having its personalized preference vector, and in the presence of context distributions that are drifting over time.