arXiv Machine Learning

Optimal Multi-Reward Reinforcement Learning

Hugging Face Trending Papers
4d ago

Introducing Feedback Delays to Non-Delayed Adversarial MDPs: Optimal $\sqrt S$ and $\sqrt K$ Dependencies in Tabular and Linear AMDPs

The paper introduces a novel technique called introduced feedback delays to address regret minimization in episodic Adversarial Markov Decision Processes (AMDPs). By estimating the Q‑function using future episodes while updating the policy, the authors achieve σO(poly(H)√{SAK}) regret in tabular AMDPs and σO(poly(d,H)√{K}) regret in linear AMDPs, resolving long‑standing open problems. The method also extends to delayed AMDPs, providing optimal bounds in terms of state count and total delay.

arXiv AI
3d ago

Introducing Feedback Delays to Non-Delayed Adversarial MDPs: Optimal $\sqrt S$ and $\sqrt K$ Dependencies in Tabular and Linear AMDPs

The paper introduces a technique called introduced feedback delays to improve regret bounds in episodic adversarial Markov Decision Processes (AMDPs). By estimating the Q‑function using future episodes while still updating the policy, the authors achieve µO(poly(H)√{SAK}) regret for tabular AMDPs and µO(poly(d,H)√{K}) regret for linear AMDPs, resolving long‑standing gaps. The method also extends to settings with actual feedback delays, providing optimal bounds in both state and delay dimensions.

By Aarush Kulkarni, Khang Nguyen, William Chang, Yan Dai