Better Policies, Worse Verifier Audits

The paper investigates how verifier components used in post‑training and test‑time loops can collapse when they are refreshed based on aggregate success scores. It shows that as a policy improves on one conjunctive target, the verifier’s on‑policy AUC can shift to favor a worse component, leading to a preference for a full‑outcome predictor over the intended component oracle. The authors provide theoretical thresholds, empirical demonstrations on text control and ECG data, and propose a repair that audits the component on its own failures to prevent damage.