arXiv Machine Learning
Bellman Calibration for Marginalized Importance Weighting in Offline Reinforcement Learning
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
Hugging Face Trending Papers
Aug 25
Bellman Calibration for Marginalized Importance Weighting in Offline Reinforcement Learning
Marginalized importance weighting evaluates a target policy by reweighting offline state-action samples with its discounted occupancy ratio, characterized by an adjoint Bellman equation. Existing mini...
arXiv Machine Learning
Jul 7
Fitted Occupancy-Ratio Evaluation without Bellman Completeness
arXiv:2607. 05375v1 Announce Type: cross Abstract: Occupancy ratios correct distribution shift in offline reinforcement learning and are central to off-policy evaluation.
Hugging Face Trending Papers
Jul 6
Fitted Occupancy-Ratio Evaluation without Bellman Completeness
Occupancy ratios correct distribution shift in offline reinforcement learning and are central to off-policy evaluation. Existing primal-dual and minimax methods typically estimate these ratios by enforcing occupancy-balance moments over a critic class.
arXiv Machine Learning
4d ago
Fitted Q-Evaluation without Bellman Completeness via Occupancy Weighting
The paper introduces occupancy-weighted Fitted Q-Evaluation (FQE), a regression-based off‑policy evaluation method that replaces the standard offline distribution weights with a target‑policy discounted occupancy ratio. This weighting aligns the projection norm with the target policy’s dynamics, restoring contraction of the Bellman operator and eliminating the need for Bellman completeness. The authors provide finite‑sample guarantees that separate iteration, statistical, approximation, and ratio‑estimation errors, and show that exact occupancy weighting combined with fitted occupancy‑ratio evaluation yields consistent estimation under coverage without requiring critic‑side completeness.
arXiv Machine Learning
4d ago
Soft Fitted Q-Iteration without Bellman Completeness: Occupancy Reweighting and Temperature Annealing
arXiv:2512.23927v3 Announce Type: replace-cross Abstract: Fitted \(Q\)-iteration (FQI) is a standard regression-based method for optimal control in offline reinforcement learning, but its stability u...
arXiv Machine Learning
Aug 5
Convex-Hull-Neighborhood Smooth Dual Generalization: Controlling Local Correction Propagation in Offline RL
arXiv:2608. 03108v1 Announce Type: new Abstract: Offline reinforcement learning (offline RL) can benefit from nearby out-of-distribution (OOD) actions, but estimation errors at these actions may be amplified by bootstrapping.