arXiv Machine Learning

Optimized Conformal Selection: Powerful Selective Inference After Data-Dependent Score Construction

The paper introduces Optimized Conformal Selection (OptCS), a framework that enables valid multiple testing after flexible, data‑driven construction of conformity scores. OptCS allows the use of all labeled data for score training and handles complex p‑value dependencies to maintain finite‑sample false discovery rate control. Three specific procedures are proposed: selecting a powerful score among candidates, training the score with all labeled data without sample splitting, and combining full‑sample training with model selection, with effectiveness demonstrated in simulations and real drug‑discovery and language‑model applications.

arXiv Machine Learning
Jul 28

Robust Conformalized Selection with Noisy Responses

arXiv:2607. 22985v1 Announce Type: cross Abstract: Conformalized selection has been widely applied to select high-quality candidates from large datasets with rigorous uncertainty quantification, such as reliable labeling, drug discovery, and the alignment of large language models.

By Chengyao Yu, Hongxin Wei, Bingyi Jing
arXiv Statistics ML
6d ago

Isotonic Conformal Inference

arXiv:2607.16675v2 Announce Type: replace Abstract: A predictor with small overall bias can still exhibit systematic bias conditional on its predicted value, limiting its reliability for downstream u...

By Daniel Bensimon, Zhiquan Liu, Sean X. Yu, Eric D. Kolaczyk, Archer Y. Yang
arXiv Machine Learning
Aug 19

Conformal Prediction for Molecular Properties under Label Shift

The paper introduces a conformal prediction framework designed for molecular property prediction under label shift. By weighting conformal scores with marginal label probability ratios, it generates statistically rigorous prediction intervals without retraining, enabling robust uncertainty quantification when property distributions change. This approach provides actionable confidence measures that improve the reliability of AI-driven predictions in drug discovery.

By Hyeonsu Lee, Juyeon Kim, Erkhembayar Jadamba, Seungjin Choi, Hyunjin Shin
arXiv Machine Learning
Sep 14

Split Conformal Prediction with Label-Shift-Adjusted Bayesian Scores

The paper introduces the Label-Shift-Adjusted Bayesian Score (LSA score), a nonconformity measure for conformal prediction that corrects Bayesian scores under label shift by applying an importance-weighted transformation of the source predictive distribution. Unlike residual-based scores that produce uniform-width intervals, the LSA score yields shorter, adaptive intervals while maintaining comparable coverage in the target domain. Experiments on molecular property prediction demonstrate that the LSA score outperforms both residual-based and source-based Bayesian scores, though all methods experience some coverage loss under stronger shifts due to density-ratio estimation challenges.

By Hyeonsu Lee, Juyeon Kim, Erkhembayar Jadamba, Seungjin Choi, Hyunjin Shin
arXiv Machine Learning
Jul 9

A Quiet Failure in Calibrated Virtual Screening: Marginal Conformal Prediction Under-Covers the Minority Class, and a Class-Conditional Fix Recovers It

arXiv:2607. 06605v1 Announce Type: new Abstract: Conformal prediction is being adopted in drug discovery to put an honest number on model reliability: pick an error rate alpha, and the method returns prediction sets containing the true label with probability at least 1 - alpha.

By Muhammadjon Tursunbadalov (School of Science and Technology, Champions College Prep, United States), Mustafojon Tursunbadalov (School of Science and Technology, Champions College Prep, United States)
arXiv Machine Learning
Jun 5

General Synthetic-Powered Inference

arXiv:2509. 20345v3 Announce Type: replace-cross Abstract: The rapid proliferation of high-quality synthetic data -- generated by advanced AI models or collected as auxiliary data from related tasks -- presents both opportunities and challenges for statistical inference.

By Meshi Bashari, Yonghoon Lee, Roy Maor Lotan, Edgar Dobriban, Yaniv Romano
arXiv Machine Learning
Sep 10

Conformalized Super Learner

arXiv:2604.22391v2 Announce Type: replace-cross Abstract: The Super Learner (SL) is a widely used ensemble method that combines point predictions from a library of learners based on their predictive...

By Zhanli Wu, Fabrizio Leisen, Miguel-Angel Luque-Fernandez, F. Javier Rubio