The paper investigates how to collect data for recalibrating a misspecified simulator that models a partially observed real-world system as a fully observed Markovian process. It compares four data‑collection strategies: never recalibrating, passive on‑policy recalibration, continuous exploration across all units, and randomized experimentation on a sample of units. Theoretical analysis and a vending‑machine simulation show that experimentation can outperform passive adaptation when drift is invisible to on‑policy data or when recalibration lags behind ongoing drift, while exploration is preferable when trying worse actions is costly and dynamics may not have changed.
arXiv:2610.02252v1 Announce Type: cross Abstract: Many scientific questions require reasoning about what was never observed: What if the conditions, interventions, or history had been different? Mode...
The paper explores how data from fixed A/B tests can guide the deployment of adaptive experiments using contextual bandits. By combining off‑policy evaluation with a controlled warm‑start simulation, the authors rank pre‑specified adaptive and non‑adaptive policies using doubly robust estimators. Experiments on synthetic trials and real benchmarks show that adaptive, context‑aware policies outperform fixed allocations when heterogeneity exists, but offer little advantage otherwise.
arXiv:2606. 22062v2 Announce Type: replace-cross Abstract: Simulation-to-reality transfer, often called sim-to-real transfer, is a central challenge in robot learning.
arXiv:2602. 05379v2 Announce Type: replace-cross Abstract: Effective reinforcement learning (RL) for complex stochastic systems requires leveraging historical data to improve sample efficiency and accelerate policy optimization.
arXiv:2512.08463v2 Announce Type: replace Abstract: We study how privileged information about a physical system affects the discovery of high-performing policies when training a reinforcement learnin...
arXiv:2609.34911v2 Announce Type: replace-cross Abstract: Modern robot policies predict a chunk of future actions from a single observation, execute only a prefix, and discard the rest before replann...
arXiv:2606. 28939v1 Announce Type: new Abstract: Behavior-cloned diffusion policies are expressive but remain vulnerable to covariate shift: small deviations from demonstrated states can compound into task failure.
arXiv:2607. 14180v1 Announce Type: cross Abstract: World models are widely used in offline reinforcement learning (RL) to improve sample efficiency and generate experience beyond a fixed dataset.
arXiv:2601. 19612v3 Announce Type: replace-cross Abstract: Safe exploration is a key requirement for reinforcement learning (RL) agents to learn and adapt online, beyond controlled (e.
arXiv:2607.08837v4 Announce Type: replace-cross Abstract: Exploration is essential to RL since a policy cannot improve by repeatedly sampling the behaviors it already prefers. Standard methods inject...
arXiv:2608. 19488v1 Announce Type: new Abstract: Production machine learning systems degrade under concept drift, yet practitioners have little principled guidance on when to retrain.
arXiv:2607. 08837v1 Announce Type: cross Abstract: Exploration is essential to RL since a policy cannot improve by repeatedly sampling the behaviors it already prefers.