Models can be retrained as new data arrive, but deploying every new version risks replacing a good policy with a worse one. We study how to plan policy updates (i.e., meta-policy) before future candid...
arXiv:2407. 21359v2 Announce Type: replace-cross Abstract: Imagining potential outcomes of actions before execution helps agents make more informed decisions, a prospective thinking ability fundamental to human cognition.
arXiv:2607. 11720v1 Announce Type: cross Abstract: Background: Offline reinforcement learning (RL) enables effective policies to be trained from large, previously collected datasets and subsequently improved through limited online interaction.
arXiv:2610.09679v1 Announce Type: new Abstract: Adaptive rollout methods for group-relative reinforcement learning typically allocate a fixed per-update budget across prompts. We instead study how to...
BCPPO is a new variant of Proximal Policy Optimization that uses Bachelier-inspired cost‑prediction networks to generate a smooth penalty based on disagreement among critics. The method keeps temporal‑difference learning unchanged, applies a saturation‑aware controller to manage cost penalties, and deploys only the policy network. Across extensive experiments, BCPPO outperforms comparators in achieving higher mean returns while maintaining lower or comparable CVaR in all tested tasks.
Background: Offline reinforcement learning (RL) enables effective policies to be trained from large, previously collected datasets and subsequently improved through limited online interaction. This offline-to-online RL (O2O-RL) paradigm is particularly promising in nonstationary domains where interaction is costly or potentially hazardous.
arXiv:2606.07399v3 Announce Type: replace-cross Abstract: Generative models are increasingly used to support decision-making in complex systems, where interventions may be joint and high-dimensional,...
arXiv:2605. 26919v2 Announce Type: replace Abstract: Maintaining predictive accuracy in non-stationary environments requires online model selection to adapt autonomously to unknown distribution shifts.
The paper investigates how a higher‑level controller in a hierarchical latent reasoner decides whether to preserve or replace a lower‑level strategy. Experiments across different training seeds show that learned revision timing can produce varied policies, but none surpasses a fixed, forced timing policy on the same checkpoint. The study highlights that state‑dependent control can alter actions without improving task performance, and that timing decisions are order‑sensitive and largely captured by a strong fixed schedule.
arXiv:2608. 01130v1 Announce Type: new Abstract: A broad range of models face the mismatch where they are updated through trajectory losses but are evaluated by downstream task reward.
The paper explores how data from fixed A/B tests can guide the deployment of adaptive experiments using contextual bandits. By combining off‑policy evaluation with a controlled warm‑start simulation, the authors rank pre‑specified adaptive and non‑adaptive policies using doubly robust estimators. Experiments on synthetic trials and real benchmarks show that adaptive, context‑aware policies outperform fixed allocations when heterogeneity exists, but offer little advantage otherwise.
arXiv:2512. 18390v2 Announce Type: replace Abstract: Organizations often have an incumbent predictive model in production when new data sources become available.