Hugging Face Trending Papers
Limiting-Kernel Q($λ$): Bridging Short and Long Horizons
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv Machine Learning
Sep 24
Limiting-Kernel Q($\lambda$): Bridging Short and Long Horizons
Limiting‑Kernel Q(λ) (LKQL) is an off‑policy value estimator that blends n‑step truncation with a long‑horizon approximation based on the limiting kernel. It maintains the computational efficiency of n‑step methods while improving policy evaluation accuracy, especially for long‑horizon tasks. The authors prove faster convergence of LKQL’s operator under aperiodicity and near‑on‑policy conditions, and demonstrate empirical gains on MuJoCo continuous‑control benchmarks.
arXiv Machine Learning
Sep 10
Decision-Centered Abstractions via Orthogonal Estimation of Difference-of-Q Functions
The paper introduces state abstractions that preserve the difference of Q‑functions for offline reinforcement learning, aiming to exclude irrelevant dynamics from rich state data. It proposes a dynamic generalization of the R‑learner that uses orthogonal estimation and sparse learning to estimate the Q‑function contrast, achieving faster convergence and consistency under a margin condition. Experiments on simulated and simulator‑augmented real data show variance reductions and demonstrate that the necessary information for sequential decision‑making can be smaller than that required for full state prediction.
arXiv Machine Learning
Aug 4
Upper-Expectile Multi-Step Q-Learning for Off-Policy Reinforcement Learning
arXiv:2608. 02034v1 Announce Type: new Abstract: Multi-step returns accelerate reward propagation in off-policy reinforcement learning, but couple the evaluation of each decision to the suboptimal logged actions that follow it, inducing a pessimistic bias that grows with the horizon.
arXiv Machine Learning
Jun 2
Trajectory Data Suffices for Statistically Efficient Policy Evaluation in Fixed-Horizon Offline RL with Linear $q^\pi$-Realizability and Concentrability
arXiv:2510. 03494v2 Announce Type: replace Abstract: We study finite-horizon offline reinforcement learning (RL) with function approximation for both policy evaluation and policy optimization.
arXiv Machine Learning
2d ago
Finite-Sample Theory for Fitted Q-Iteration When Actions Are Functions
arXiv:2609.36390v1 Announce Type: cross Abstract: Offline reinforcement learning seeks optimal decision rules from previously collected data. In some applications, a decision can be an entire functio...
arXiv Machine Learning
Aug 24
Smart Exploration in Reinforcement Learning using Bounded Uncertainty Models
The paper introduces BUMEX, a reinforcement learning exploration strategy that leverages a set of prior models containing the true transition kernel and reward function. By optimizing over this model set, the method derives upper and lower bounds on the Q‑function to guide exploration, providing theoretical guarantees of convergence to the optimal policy. When the model set follows a bounded‑parameter MDP structure, the optimization becomes convex, enabling finite‑time convergence under mild assumptions and demonstrating accelerated learning in simulations.