arXiv Machine Learning
Exponential Hardness of Off-Policy Evaluation under History-Dependent Logging
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Aug 26
Model-Based Learning of Near-Optimal Finite-Window Policies in POMDPs
arXiv:2604.01024v2 Announce Type: replace Abstract: We study model-based learning of finite-window policies in tabular partially observable Markov decision processes (POMDPs). A common approach to le...
arXiv Machine Learning
Jun 2
Online Learning in MDPs with Partially Adversarial Transitions and Losses
arXiv:2602. 09474v2 Announce Type: replace Abstract: We study reinforcement learning in MDPs whose transition function is stochastic at most steps but may behave adversarially at a fixed subset of $\Lambda$ steps per episode.
arXiv Machine Learning
Jun 3
Minimax Optimal Strategy for Delayed Observations in Online Reinforcement Learning
arXiv:2603. 03480v2 Announce Type: replace Abstract: We study reinforcement learning with delayed state observation, where the agent observes the current state after some random number of time steps.
arXiv Machine Learning
Jun 11
Reinforcement Learning with Action-Triggered Observations
arXiv:2510. 02149v2 Announce Type: replace Abstract: We introduce Action-Triggered Sporadically Traceable Markov Decision Processes (ATST-MDPs), a reinforcement learning framework for partial observability in which full state observations occur stochastically at each step, with probability determined by the chosen action.
arXiv AI
Aug 26
Knowing When to Ask for Help: Bayesian Self-Escalation in Hierarchical LLM Agents
The paper introduces a Bayesian self‑escalation strategy for hierarchical large‑language‑model agents, allowing an agent to detect during its own reasoning that it is unlikely to succeed and hand control over to a stronger model. The authors formalise this as an optimal‑stopping problem over a learned competence posterior, derive a myopic escalation threshold, and prove that the optimal policy is a time‑varying threshold without assumptions on the raw signal. They provide theoretical guarantees—including a 1/√n regret decay with n calibration trajectories—and validate the approach in simulations and a real‑model code‑generation cascade, showing that the escalation frontier outperforms post‑hoc routing at equal cost. whyItMatters":"The study offers a principled, theoretically grounded method for agents to dynamically decide when to seek stronger models, potentially improving efficiency and reliability in hierarchical LLM systems."
arXiv AI
Jul 21
Expected Free Energy as Belief-Dependent Utility for rho-POMDPs
arXiv:2607. 16981v1 Announce Type: new Abstract: An agent acting under partial observability must decide when to gather information and which observations are worth their cost.