arXiv:2609.36812v1 Announce Type: cross Abstract: Diffusion and flow policies can model complex behaviors in offline reinforcement learning (RL). However, penalizing their KL divergence from the beha...
arXiv:2608. 01205v1 Announce Type: new Abstract: Recent offline reinforcement learning methods increasingly rely on expressive generative policies and specialized value-guidance mechanisms.
The paper introduces Bidirectional Behavior Prior Distillation (B2PD), a method that uses action‑value priors to train a conditional variational autoencoder for generating high‑value behavior support. These expert behavior priors are then distilled into the online reinforcement learning agent, reducing inefficient exploration and stabilizing policy updates. Experiments on state‑ and pixel‑based tasks show that B2PD improves sample efficiency while maintaining stable learning dynamics.
arXiv:2601. 00898v3 Announce Type: replace Abstract: Diffusion-based policies have gained growing popularity in solving a wide range of decision-making tasks due to their superior expressiveness and controllable generation during inference.
arXiv:2609.15123v1 Announce Type: cross Abstract: Flow-based policies offer an expressive representation for online reinforcement learning, but conventional flow matching requires samples drawn from...
arXiv:2605. 03065v2 Announce Type: replace Abstract: Generative control policies (GCPs), such as diffusion- and flow-based control policies, have emerged as effective parameterizations for robot learning.
arXiv:2606. 09115v1 Announce Type: new Abstract: Offline reinforcement learning (RL) offers a path to policy improvement from logged data alone, using historical returns or other measurable outcomes as world feedback.
arXiv:2608.20909v1 Announce Type: new Abstract: Offline RL methods commonly jointly train the actor and critic, where the critic is used to guide the actor toward higher-value actions. This coupled l...
The paper introduces Multi-step Proximal Policy Improvement (MPI), a method that refines offline reinforcement learning policies through sequential re-centered proximal steps. By modeling policies as a probability manifold, MPI interprets a wide range of offline actor objectives as a single proximal policy improvement step and extends this to multiple steps for controlled policy improvement beyond the behavior distribution. Experiments on D4RL benchmarks demonstrate that a few MPI refinements enhance strong offline baselines such as TD3+BC, ReBRAC, and IQL across many tasks, while diagnostics clarify the benefits of re-centered refinement over fixed-objective scheduling and highlight critic error limitations.
The paper introduces Multi-step Proximal Policy Improvement (MPI), a method that refines offline reinforcement learning policies through sequential re-centered proximal steps. By viewing policies as a probability manifold, MPI interprets a wide range of offline actor objectives as a single proximal policy improvement step and extends this to multiple steps for controlled policy improvement beyond the behavior distribution. Experiments on D4RL benchmarks demonstrate that a few MPI refinements enhance strong offline baselines such as TD3+BC, ReBRAC, and IQL, while diagnostics clarify the benefits of re-centered refinement over fixed-objective scheduling and highlight critic error limitations.
arXiv:2606. 05888v1 Announce Type: new Abstract: Retry-based objectives such as pass@K and max@K optimize the best return obtained from multiple sampled trajectories, and recent work has shown that they can promote exploration without explicit exploration bonuses.
arXiv:2607. 21302v1 Announce Type: new Abstract: Behavior prior reinforcement learning (BPRL) has emerged as a promising paradigm to improve sample efficiency in online reinforcement learning (RL) by leveraging policy priors derived from offline demonstrations.