The paper introduces Adaptive Doubly Robust (ADR), an off‑policy evaluation method for ranking policies that blends adaptive importance weighting with reward regression to reduce variance. ADR is unbiased when the true user behavior model is known and, under a sufficient condition, achieves lower variance than the prior Adaptive Inverse Propensity Scoring (AIPS) approach. Experiments on synthetic data show that ADR consistently improves mean squared error over AIPS and other ranking OPE estimators across various data sizes and ranking lengths.
By Kosuke Iguchi, Ren Kishimoto
arXiv:2610.08677v1 Announce Type: new
Abstract: Off-policy evaluation (OPE) for contextual bandit policies becomes challenging when action-level importance weighting incurs excessive variance. Doubly...
By Nicol\`o Felicioni, Michael Benigni, Maurizio Ferrari Dacrema, Paolo Cremonesi
Off-policy evaluation (OPE) for contextual bandit policies becomes challenging when action-level importance weighting incurs excessive variance. Doubly robust (DR) estimation remains unbiased under co...
arXiv:2606. 19117v1 Announce Type: cross Abstract: Offline policy learning has received growing attention in causal inference.
By Yiyan Huang, Cheuk Hang Leung, Qi Wu, Zhiheng Zhang
The paper introduces a doubly robust framework for distributionally robust offline contextual pricing, addressing challenges of distributional shifts and continuous actions. It proposes a localized DR estimator that is computationally efficient and semiparametrically optimal, and a learning algorithm that exploits demand noise smoothness to achieve finite‑sample regret bounds of τ^{-σ/(2σ+1)} for smoothness orders s=1,2. Experiments across varying distribution shifts demonstrate the effectiveness of the proposed methods.
By Min Xu, Xinyi Yin, Yunfan Zhang, Yuxuan Han, Houcai Shen, Caihua Chen
The paper introduces Prediction-Powered Ranking (PP‑Rank), a framework that blends large language model (LLM) judge predictions with limited human feedback to rank LLMs more precisely while maintaining valid inference under adaptive sampling. PP‑Rank uses data‑driven prediction weights to minimize asymptotic covariance, establishes asymptotic normality for general evaluation targets, and provides a rule for reducing exploration that balances cost and variance. Experiments on UltraFeedback‑Extended and synthetic data show that PP‑Rank improves score estimation, narrows confidence intervals, certifies rankings, and can cut evaluation costs without losing precision.
By Gabriel Sargent, Will Wei Sun, Zhengwu Zhang, Yufeng Liu
arXiv:2609.36740v1 Announce Type: new
Abstract: Many recommender systems such as for e-commerce and news platforms aim to provide users with rankings they are likely to interact with. Off-Policy Lear...
By Ren Kishimoto, Koichi Tanaka, Haruka Kiyohara, Yusuke Narita, Yasuo Yamamoto, Nobuyuki Shimizu, Yuta Saito
The paper introduces a sample‑only framework for evaluating and selecting Best‑of‑N (BoN) inference‑time alignment policies without needing response likelihoods. By exploiting the order‑statistic structure of BoN, the authors express required density ratios through score‑rank probabilities that can be estimated from samples alone, and develop a doubly robust estimator (BoN‑DR) that reuses a shared auxiliary sample pool across candidate budgets. The method provides valid asymptotic inference even with reward estimator misspecification, proves efficiency, and offers two selection rules—maximizing estimated policy value or a lower confidence bound—to avoid over‑optimization and guarantee no harm, as demonstrated on synthetic data and GSM8K.
By Jonas Schweisthal, Yuxin Wang, Athiya Deviyani, Stefan Feuerriegel, Dennis Frauen
arXiv:2610.32910v1 Announce Type: new
Abstract: Regularised reinforcement learning (RL) algorithms rely on estimators, like Monte Carlo tree search or one-step look-ahead, to provide policy, value, o...
By Danilo Vucetic, Alexandre Larouche, Mohammad Pedramfar, Audrey Durand, Gauthier Gidel
arXiv:2509. 03456v2 Announce Type: replace-cross Abstract: Off-policy evaluation (OPE) and off-policy learning (OPL) are foundational for decision-making in offline contextual bandits.
By Imad Aouali, Otmane Sakhi
arXiv:2606. 03332v1 Announce Type: new Abstract: Probabilistic models are typically trained using task-agnostic objectives like log-loss, which can lead to significant errors in downstream estimation.
By Roman Plaud, Alexandre Perez-Lebel, Antoine Saillenfest, Thomas Bonald, Marine Le Morvan, Ga\"el Varoquaux, Matthieu Labeau
The paper proposes a method for allocating limited counterfactual annotation resources from multiple sources—such as domain experts and large language models—to improve off-policy evaluation in contextual bandits. By formulating an integer optimization problem that balances source-specific costs and error rates, the authors derive a majorization‑minimization algorithm that iteratively reduces estimator variance. Experiments on synthetic clinical and education bandit data demonstrate that the optimized allocation cuts mean‑squared error by over 20% compared to no annotation.
By Biao Xiang, Ali Eshragh, Yuexing Li, Kai Wang