arXiv Statistics ML

Model Predictive Control is almost Optimal for Heterogeneous Restless Multi-armed Bandits

arXiv Machine Learning

Oct 7

Linear Bandits under Exact Sliding-Window Constraints

arXiv:2610. 08745v1 Announce Type: new Abstract: We study linear bandits under exact sliding-window constraints, where every consecutive block of actions must belong to a prescribed feasible set.

By Seyed Mohammad Hadi Hosseini, Yasin Abbasi-Yadkori, Sattar Vakili

benchmarks

arXiv Machine Learning

Sep 30

Nonpreemptive Scheduling While Learning Context-Dependent Service Rates

arXiv:2609.37660v1 Announce Type: new Abstract: We study nonpreemptive contextual queueing bandits in a single-server system. Each job is represented by a $d$-dimensional context vector; in each roun...

By Wansoo Choi, Seoungbin Bae, Dabeen Lee

arXiv Machine Learning

Oct 2

Rising Multi-Armed Bandits with Known Horizons

arXiv:2602.10727v3 Announce Type: replace Abstract: Rising Multi-Armed Bandits (RMABs) model sequential decision problems where each arm's expected reward improves with repeated pulls. In such proble...

By Seockbean Song, Chenyu Gan, Youngsik Yoon, Siwei Wang, Wei Chen, Jungseul Ok

reinforcement-learning benchmarks

arXiv Machine Learning

Oct 8

Online Resource Allocation with an Endogenous Markov State: Fewer LP Solves Earn More

arXiv:2610.09577v1 Announce Type: new Abstract: We study finite-horizon online resource allocation with i.i.d. requests and an endogenous Markov state on a finite state space: each action affects the...

By Zhaohua Chen

benchmarks

arXiv Machine Learning

Sep 22

Multi-Armed Bernoulli Bandits via Minimax Single-Arm Stopping

arXiv:2609. 22690v1 Announce Type: new Abstract: We develop an index policy for finite-horizon Bernoulli multi-armed bandits from minimax solutions to single-arm bandit (SAB) problems.

By Huikang Liu, Zhengchao Wang, Daniel Kuhn, Wolfram Wiesemann

reinforcement-learning benchmarks

arXiv Machine Learning

Jun 29

Learning in Markovian bandits with non-observable states and constrained decision epochs

arXiv:2606. 27448v1 Announce Type: new Abstract: This paper studies the problem of regret minimization in Markovian bandits with \emph{non-observable states} and possibly \emph{constrained} decision epochs.

By Thomas Hira, Victor Boone, Urtzi Ayesta, Ina Maria Verloop

reinforcement-learning benchmarks safety

arXiv Machine Learning

Sep 30

Achieving an $O(1/N)$ Optimality Gap in Average-Reward Weakly-Coupled MDPs

arXiv:2609.38132v1 Announce Type: new Abstract: We study average-reward weakly-coupled Markov decision processes (WCMDPs), where a WCMDP consists of $N$ smaller MDPs, called arms, that share multiple...

By Yige Hong, Xiangcheng Zhang, Qiaomin Xie, Yudong Chen, Weina Wang

reinforcement-learning

arXiv Machine Learning

Jul 28

Finite-Time Analysis of the Natural Policy Gradient in Finite-Horizon Markov Decision Processes

arXiv:2607. 22982v1 Announce Type: new Abstract: Natural Policy Gradient (NPG) is a well-established Reinforcement Learning algorithm that underlies widely used methods such as Trust Region Policy Optimization and Proximal Policy Optimization, both of which have demonstrated strong empirical success.

By Asha Barua, Sajad Khodadadian

reinforcement-learning

arXiv Statistics ML

Oct 2

Block Optimism for Nonstationary Bandits with Latent Linear Dynamics

The paper introduces a new algorithm for a nonstationary bandit setting where actions influence both immediate rewards and the evolution of an unobserved latent linear state. By approximating the infinite‑memory reward process with a finite‑memory block‑level proxy and applying a UCB‑based block algorithm, the authors achieve a regret bound of “~O(√T)”, improving upon the previous “~O(T^{2/3})” guarantee. This represents the first such “~O(√T)” result for latent linear‑dynamics bandits with bilinear rewards and an open‑loop action‑sequence benchmark.

By Taehyun Hwang, Hyunjun Choi, Heesang Ann, Min-hwan Oh

reinforcement-learning benchmarks

arXiv Machine Learning

Jul 23

Asymptotically Optimal Regret for Reinforcement Learning without Horizon Dependence

arXiv:2607. 19854v1 Announce Type: new Abstract: We study horizon-free regret minimization for finite-horizon time-homogeneous tabular Markov decision processes with $S$ states, $A$ actions, horizon $H$, and per-trajectory total reward bounded by $1$.

By Runlong Zhou, Zihan Zhang, Maryam Fazel, Simon S. Du

reinforcement-learning

arXiv Machine Learning

Jun 15

Lyapunov-Based Sample Complexity Analysis for Weakly-Coupled MDPs

arXiv:2606. 14095v1 Announce Type: new Abstract: We study the sample complexity of learning in average-reward weakly-coupled Markov decision processes (WCMDPs) and Restless Bandits (RBs) under a generative model.

By Tianhao Wu, Matthew Zurek, Weina Wang, Qiaomin Xie

reinforcement-learning safety

arXiv Machine Learning

3d ago

Transfer Learning in Bandits from Unknown Adaptive Policies: Problem-Dependent Lower and Upper Bounds

The paper investigates transfer learning in stochastic multi‑armed bandits when historical data come from an unknown adaptive policy. It establishes asymptotic problem‑dependent lower bounds and introduces the KL‑UCB‑Transfer algorithm, which combines online and historical data to achieve logarithmic regret with an improved constant under certain mean‑gap conditions. The study also relaxes the source‑target discrepancy assumption to a one‑sided condition, demonstrating that logarithmic regret can still be achieved under misspecification and identifying regimes of improvement, degradation, or loss of guarantee.

By Adrien Prevost, Timothee Mathieu, Waris Radji, Odalric-Ambrym Maillard

fine-tuning

Related stories