← Back to all news
arXiv Machine Learning By Huizhen Yu, Isaiah Heidt

Average-Reward Reinforcement Learning for Multichain MDPs: A Hierarchical Decomposition Approach

Read the original on arXiv Machine Learning →

The paper introduces an asynchronous value‑iteration RL algorithm for average‑reward multichain MDPs that requires only the transition graph and uses Bather’s decomposition to partition the state space into communicating subsystems and transient states. This hierarchical decomposition transforms the global decision problem into structured subproblems, enabling the algorithm to converge to the optimal gain and produce gain‑optimal policies in finite time. Two additional algorithms are presented: one that approximately solves the multichain average optimality equations for near gain‑optimality, and another that approximates the optimal bias function to achieve near bias‑optimality, both with almost‑sure convergence guarantees and improved transient performance. whyItMatters":"The work offers the first essentially model‑free average‑reward RL algorithms for general multichain MDPs, avoiding reductions to discounted problems."

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Jun 16

Learning Policy from a Single Trajectory in Average-Reward Markov Decision Process

arXiv:2606. 16729v1 Announce Type: new Abstract: While there is an extensive body of work characterizing the sample complexity of discounted cumulative-reward MDPs, finite sample analyses for average-reward MDPs have been limited, and most existing works rely on restrictive assumptions such as ergodicity or access to a generative model.

By Jongmin Lee, Ernest K. Ryu, Vaneet Aggarwal
reinforcement-learning
More like this →
arXiv Machine Learning
Jul 1

End-to-End Efficient RL for Linear Bellman Complete MDPs with Deterministic Transitions

arXiv:2603. 23461v2 Announce Type: replace Abstract: We study reinforcement learning (RL) with linear function approximation in Markov Decision Processes (MDPs) satisfying \emph{linear Bellman completeness} -- a fundamental setting where the Bellman backup of any linear value function remains linear.

By Zakaria Mhammedi, Alexander Rakhlin, Nneka Okolo
reinforcement-learning
More like this →
arXiv Machine Learning
3d ago

Reinforcement Learning in MDPs with Information-Ordered Policies

The paper introduces a framework for reinforcement learning in Markov decision processes where policies are ordered by the information they provide, allowing data from one policy to estimate the performance of others. An epoch‑based algorithm is developed for infinite‑horizon average‑cost MDPs with continuous state and action spaces, achieving regret that scales with the width of the policy class’s information order. The authors apply the framework to lost‑sales inventory, dual‑sourcing, and state‑dependent queueing systems, demonstrating improved regret bounds and empirical performance without requiring global convexity or specific arrival‑rate structures.

By Zhongjun Zhang, Shipra Agrawal, Ilan Lobel, Sean R. Sinclair, Christina Lee Yu
reinforcement-learningbenchmarkssafety
More like this →
arXiv Machine Learning
Oct 3

Towards Optimal Policy Improvement

The paper introduces a framework for optimal policy improvement in reinforcement learning, defining it as the best single update under given constraints. It shows that restricting improvement to a subset of states is equivalent to solving an induced Markov Decision Process, linking planning with explicit or implicit models to optimal policy improvement. The authors develop a novel operator for greedification under approximate evaluation, demonstrating empirical gains across several RL algorithms and settings.

By Yaniv Oren, Viliam Vadocz, Wiktor Zabka, Thomas Evers, Jan Robine, Wendelin B\"ohmer, Matthijs T. J. Spaan, Martha White, Hendrik Baier, Fenghui Yu
reinforcement-learning
More like this →