The paper extends dueling bandits to episodic Markov decision processes where selecting a pair of arms not only yields a noisy preference but also influences the next state. It defines a state‑wise preference‑based MDP with adversarially changing preferences and Borda‑score losses, and proves a minimax regret of ∫Θ(min{HT,(H^3SK)^{1/3}T^{2/3}}) under known transitions. The authors provide an online linear‑optimization algorithm achieving this bound, a policy‑optimization algorithm with closed‑form updates, and extensions to unknown transitions.
arXiv:2602. 09474v2 Announce Type: replace Abstract: We study reinforcement learning in MDPs whose transition function is stochastic at most steps but may behave adversarially at a fixed subset of $\Lambda$ steps per episode.
The paper addresses policy optimization in online episodic tabular MDPs with adversarial losses and bandit feedback. By employing regularized Q‑functions, the authors eliminate the extra horizon factor present in previous policy‑optimization regret bounds, achieving high‑probability regret of ≥O(√{HS(H+A)T}) for known transitions and ≥O(HS√{AT}) for unknown transitions. These results improve horizon dependence and match the best-known bounds, with extensions to adversarial linear‑mixture MDPs.
arXiv:2609.13547v1 Announce Type: new Abstract: We study switching regret in adversarial multi-armed bandits, where the learner competes with an arm sequence that changes at most $S$ times. When $S$...
The paper introduces the first algorithms for replicable dueling bandits, ensuring that two runs with shared internal randomness and independent comparison outcomes select the same ordered pair in every round with high probability. Under relaxed stochastic transitivity, the authors achieve expected regret ×O(γ^8 K^4 H_p/ρ^2), while under strong stochastic transitivity they improve this to ×O(K^2 H_p/ρ^2). For linear dueling bandits, a regret bound of ×O(d^3√T/(c_μρ)) is presented, and experiments confirm both replicability and sublinear regret.
arXiv:2609. 15248v2 Announce Type: replace-cross Abstract: In bandits with probing, a learner selects at most $k$ of $n$ arms per round, earns the largest of their rewards, and competes with the best fixed single arm.
arXiv:2608. 09501v1 Announce Type: new Abstract: We consider the extensive-form bandit problem where on each trial the learner plays an extensive-form game against an oblivious adversary.
arXiv:2606. 27448v1 Announce Type: new Abstract: This paper studies the problem of regret minimization in Markovian bandits with \emph{non-observable states} and possibly \emph{constrained} decision epochs.
arXiv:2610. 30801v1 Announce Type: new Abstract: We study episodic linear Markov decision processes with unknown transitions, adversarial losses, and bandit feedback.
The paper studies the trade‑off between maximizing rewards and accurately estimating current treatment effects in nonstationary multi‑armed bandits. For $K$‑armed bandits with H"older‑smooth mean rewards, the authors derive minimax bounds on expected dynamic regret under a high‑probability simultaneous coverage requirement for all current effects. They propose the TRACE algorithm, which achieves a regret–precision trade‑off that separates reward‑learning regret from the unavoidable cost of refreshing current‑effect estimates, and extend the method with an empirical‑Bernstein e‑process to adapt exploration while preserving worst‑case guarantees.
arXiv:2610. 12153v1 Announce Type: new Abstract: We study episodic constrained Markov decision processes with adversarial losses under stochastic hard constraints.
arXiv:2605. 01961v2 Announce Type: replace Abstract: Learning from human preference data is becoming a useful tool, from fine-tuning large language models to training reinforcement learning agents.
arXiv:2606. 19883v1 Announce Type: new Abstract: We study a multi-agent multi-armed bandit problem in the competitive setup with two-sided matching markets under a human centric decision making model.