Budgeted Multi-Source Counterfactual Annotation for Off-Policy Evaluation

The paper proposes a method for allocating limited counterfactual annotation resources from multiple sources—such as domain experts and large language models—to improve off-policy evaluation in contextual bandits. By formulating an integer optimization problem that balances source-specific costs and error rates, the authors derive a majorization‑minimization algorithm that iteratively reduces estimator variance. Experiments on synthetic clinical and education bandit data demonstrate that the optimized allocation cuts mean‑squared error by over 20% compared to no annotation.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• llms
• reinforcement-learning