arXiv:2610. 30801v1 Announce Type: new Abstract: We study episodic linear Markov decision processes with unknown transitions, adversarial losses, and bandit feedback.
By Haolin Liu, Chen-Yu Wei, Hanbin Zhou
arXiv:2602. 09456v2 Announce Type: replace Abstract: We propose an algorithmic framework, Offline Estimation to Decisions (OE2D), that efficiently reduces contextual bandit learning with general reward function approximation to offline regression.
By Hao Qin, Chicheng Zhang
The paper introduces a hierarchical Bayesian framework for multi-task contextual bandits, capturing both heterogeneity and correlations among bandit instances. It proposes an empirical Bayesian method to estimate the covariance matrix of the prior, leading to two new algorithms—ebmTS and ebmUCB—that incorporate this prior into decision making. The authors provide frequentist regret bounds and show through synthetic and real-world experiments that these algorithms outperform existing methods, especially in complex settings.
By Xia Jiang, Rong J. B. Zhu
arXiv:2004. 06321v2 Announce Type: replace Abstract: We study the sequential batch learning problem in linear contextual bandits with finite action sets, where the decision maker is constrained to split incoming individuals into (at most) a fixed number of batches and can only observe outcomes for the individuals within a batch at the batch's end.
By Yanjun Han, Zhengqing Zhou, Zihao Hu, Jose Blanchet, Peter W. Glynn, Yinyu Ye, Zhengyuan Zhou
The paper investigates stochastic contextual bandits in an agnostic setting, aiming to compete with the best policy in a given class without assuming realizability or specific loss/reward models. It introduces an algorithm that updates the policy each round by minimizing a pessimistic objective— a clipped inverse‑propensity estimate of the policy value plus a variance penalty— and proves the first fast regret rates relative to the best‑in‑class policy. By exploiting entropy assumptions on the policy class and a H"olderian error‑bound condition, the authors achieve fast best‑in‑class regret rates, including polylogarithmic rates in the parametric case, using a sequential self‑normalized maximal inequality for bounded martingale empirical processes to derive uniform variance‑adaptive confidence bounds and ensure pessimism under adaptive data collection.
By Samuel Girard, Aurelien Bibaut, Arthur Gretton, Nathan Kallus, Houssam Zenati
arXiv:2607. 28408v1 Announce Type: new Abstract: This thesis studies policy learning in interactive systems where an agent observes a context, selects an action from a very large set, and receives partial feedback.
By Imad Aouali