arXiv Machine Learning July 8, 2026 By Ziwei Su, Imon Banerjee, Diego Klabjan

Model-based Bootstrap of Controlled Markov Chains

Read the original on arXiv Machine Learning →

arXiv:2605. 12410v2 Announce Type: replace-cross Abstract: We propose and analyze a model-based bootstrap for transition kernels in finite controlled Markov chains (CMCs) with possibly nonstationary or history-dependent control policies, a setting that arises naturally in offline reinforcement learning (RL) when the behavior policy generating the data is unknown.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

reinforcement-learning

arXiv Machine Learning

Jun 15

PERRY: Policy Evaluation with Confidence Intervals using Auxiliary Data

arXiv:2507. 20068v2 Announce Type: replace Abstract: Off-policy evaluation (OPE) methods estimate the value of a new reinforcement learning (RL) policy prior to deployment.

By Aishwarya Mandyam, Jason Meng, Ge Gao, Jiankai Sun, Mac Schwager, Barbara E. Engelhardt, Emma Brunskill

reinforcement-learning robotics

arXiv Machine Learning

2d ago

Online Inference in Distributional Temporal-Difference Learning

arXiv:2608. 14408v1 Announce Type: cross Abstract: We study online statistical inference for functionals of the return distribution under a fixed policy.

By Yang Peng, Liangyu Zhang

arXiv AI

Jun 18

Robust Regularized Policy Iteration under Transition Uncertainty

arXiv:2603. 09344v3 Announce Type: replace Abstract: Offline reinforcement learning (RL) enables data-efficient and safe policy learning without online exploration, but its performance often degrades under distribution shift.

By Hongqiang Lin, Zhenghui Fu, Weihao Tang, Pengfei Wang, Yiding Sun, Qixian Huang, Dongxu Zhang

reinforcement-learning benchmarks

arXiv Machine Learning

Aug 12

Generalized Linear Markov Decision Process

arXiv:2506. 00818v2 Announce Type: replace-cross Abstract: Offline reinforcement learning for longitudinal studies often faces two linked challenges: rewards may be binary or bounded, and reward observations may be available only for a subset of trajectories or time points even when the corresponding state-action-next-state histories are available.

By Sinian Zhang, Kaicheng Zhang, Ziping Xu, Zongqi Xia, Jue Hou, Tianxi Cai, Doudou Zhou

reinforcement-learning

arXiv Machine Learning

Jun 18

When Does Trajectory-Level Supervision Permit Efficient Offline Reinforcement Learning?

arXiv:2606. 18531v1 Announce Type: cross Abstract: Offline reinforcement learning is typically analyzed under process-level reward supervision, yet many sequential decision datasets record only trajectory-level outcomes.

By Xuanfei Ren, Tengyang Xie

reinforcement-learning

arXiv Machine Learning

Aug 5

Convex-Hull-Neighborhood Smooth Dual Generalization: Controlling Local Correction Propagation in Offline RL

arXiv:2608. 03108v1 Announce Type: new Abstract: Offline reinforcement learning (offline RL) can benefit from nearby out-of-distribution (OOD) actions, but estimation errors at these actions may be amplified by bootstrapping.

By Yi Yang, Zhennan Chen, Mingfeng Lv, Hanlei Li, Zhengsen Ruan, Lvqing Yang

reinforcement-learning

Related stories

PERRY: Policy Evaluation with Confidence Intervals using Auxiliary Data

Online Inference in Distributional Temporal-Difference Learning

Robust Regularized Policy Iteration under Transition Uncertainty

Generalized Linear Markov Decision Process

When Does Trajectory-Level Supervision Permit Efficient Offline Reinforcement Learning?

Convex-Hull-Neighborhood Smooth Dual Generalization: Controlling Local Correction Propagation in Offline RL