arXiv AI
Jul 23
Sophisticated Policies from Epistemic Priors
arXiv:2607. 19518v1 Announce Type: new Abstract: Sophisticated Inference is a variant of active inference often associated with recursive belief modeling and tree search.
Active Inference as a Convex Markov Decision Process
arXiv:2607. 20152v1 Announce Type: cross Abstract: Active Inference (AIF) frames adaptive behavior as the minimization of expected free energy (EFE), combining epistemic and pragmatic objectives within a single variational principle.
Related stories
arXiv AI
Jul 10
Second-Order Actor-Critic Methods for Discounted MDPs via Policy Hessian Decomposition
arXiv:2605. 14982v2 Announce Type: replace-cross Abstract: We address the discounted reward setting in reinforcement learning (RL).
arXiv Machine Learning
Jul 9
Mathematical methods of reinforcement learning
arXiv:2607. 06935v1 Announce Type: cross Abstract: Reinforcement learning (RL) is increasingly grounded in tools from probability, optimization, and operator theory.
arXiv Machine Learning
1d ago
Towards Optimal Policy Improvement
The paper introduces a framework for optimal policy improvement in reinforcement learning, defining it as the best single update under given constraints. It shows that restricting improvement to a subset of states is equivalent to solving an induced Markov Decision Process, linking planning with explicit or implicit models to optimal policy improvement. The authors develop a novel operator for greedification under approximate evaluation, demonstrating empirical gains across several RL algorithms and settings.
arXiv AI
Jun 4
What Type of Inference is Active Inference?
arXiv:2606. 04935v1 Announce Type: new Abstract: Active inference casts decision-making as inference, with the Expected Free Energy (EFE) unifying goal-directed and information-seeking behavior.
arXiv AI
4d ago
Reward-rate Policy Gradient for Efficient Machine Learning Engineering Agents
arXiv:2609.36393v1 Announce Type: cross Abstract: Traditional reinforcement learning (RL) techniques focus on maximizing expected cumulative reward, where each action assumes to take a constant unit...
arXiv AI
Jun 10
Bellman-Taylor Score Decoding for Markov Decision Processes with State-Dependent Feasible Action Sets
arXiv:2606. 10979v1 Announce Type: new Abstract: Many Markov decision processes (MDPs) in operations research have feasible actions that are state dependent and defined implicitly by various operational constraints.
arXiv Machine Learning
Sep 15
Learning to Solve Stochastic Controls with Unknown Drifts and Running Rewards: Theory, Algorithms and Convergence
The paper investigates continuous‑time stochastic control problems with unknown drift and running reward functions, using an exploratory reinforcement learning framework that incorporates relaxed controls and entropy regularization. It develops policy‑iteration algorithms based on probabilistic representations of the optimal value function and its gradient, proving convergence and demonstrating performance through numerical examples. The study also extends to a special case with control‑dependent diffusion, requiring a Hessian representation.
arXiv Machine Learning
Sep 24
Value Mirror Descent for Reinforcement Learning
arXiv:2604. 06039v2 Announce Type: replace-cross Abstract: Value iteration-type methods have been extensively studied for computing a nearly optimal value function in reinforcement learning (RL).
arXiv Machine Learning
4d ago
Global Optimality for Constrained Exploration via Penalty Regularization
The paper introduces Policy Gradient Penalty (PGP), a single‑loop policy‑space method that enforces convex occupancy‑measure constraints via quadratic‑penalty regularization. PGP constructs pseudo‑rewards to estimate gradients of the penalized objective and uses the classical Policy Gradient Theorem, establishing smoothness and global last‑iterate convergence guarantees for an ε‑optimal constrained entropy value with ε‑bounded constraint violation. The authors validate PGP with ablations on a grid‑world benchmark and demonstrate scalability on two challenging continuous‑control tasks.
arXiv AI
Jun 19
Can In-Context Learning Support Intrinsic Curiosity?
arXiv:2606. 19476v1 Announce Type: cross Abstract: Effective machine learning depends not only on how we model data, but also on what data we choose to collect.
arXiv AI
Jun 30
Exploration and Online Transfer with Behavioral Foundation Models
arXiv:2606. 29980v1 Announce Type: new Abstract: Zero-shot Transfer in Reinforcement Learning (RL) aims to train an agent that can generate optimal policies for any reward function, without additional learning at transfer time, while training only on reward-free trajectories.