Before the Next Rollout: Diagnosing Skill Utility Shifts from Policy Updates

The paper investigates how policy updates can alter the marginal utility of unchanged skills in reinforcement learning agents. By analyzing changes in token preferences between skill and control conditions at fixed contexts from pre‑update rollouts, the authors develop readouts that combine interaction shifts with advantage estimates to diagnose utility changes. These readouts use only forward passes on existing rollouts, avoiding extra post‑update evaluations, and demonstrate that incorporating them into skill evolution improves downstream performance and reduces token consumption.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• agents