The paper investigates when conformal calibration for cause‑of‑failure prediction sets under competing risks requires censoring weights. It shows that using only complete‑case calibration can lead to coverage that is either above or below the nominal level, especially when a substantial proportion of subjects are event‑free at the prediction horizon. The authors demonstrate through simulations that incorporating weights from a correctly specified censoring model restores nominal coverage, and they provide a finite‑sample lower bound that penalises censoring‑model error, highlighting the risk of under‑coverage when the model is misspecified.
By Sunny Yang, Weiyan Zhao
arXiv:2610.09541v1 Announce Type: cross
Abstract: Signals that predict whether a chain-of-thought (CoT) trace is correct are compared by AUC, but deploying one requires a threshold with a guarantee....
By Arjun Balaji
The paper evaluates hard‑gate candidacy for validators in a deployed generative‑agent system by measuring how well each validator’s firing separates successful from failed builds. Across 13 validators and thousands of builds, only a few checks show statistically significant separation, while many fail to distinguish or never fire. The study highlights that skipped checks are recorded as passes, limiting detectable failure rates and underscoring the need for clearer evaluation records.
By Xin Xu
arXiv:2609.38917v1 Announce Type: new
Abstract: A classifier's conditional accuracy can change while its confidence distribution stays exactly the same. We study the worst-case movement of the reliab...
By Wenhao Liang, Lin Yue, Wei Emma Zhang, Mingyu Guo, Olaf Maennel, Weitong Chen
arXiv:2608.23638v1 Announce Type: cross
Abstract: Two analysts who calibrate the same predictive model on independent samples will deploy different prediction sets every time, because the calibration...
By Marios Papamichalis, Regina Ruane, Theofanis Papamichalis
arXiv:2608. 12489v1 Announce Type: new Abstract: Organizations decide whom to treat under a budget and want to know what a targeting rule would have earned before deploying it.
By Binshuang Li
arXiv:2606. 29654v1 Announce Type: new Abstract: Multi-agent deliberation among LLMs can improve reasoning, but deployment requires deciding when the current answer is reliable enough to act on and when it should be escalated to human review.
By Mengdie Flora Wang, Haochen Xie, Guanghui Wang, Devin Zhang, Jae Oh Woo
The paper introduces a method to certify selective prediction in machine learning systems by computing the availability of safety gates through exact-binomial inversion and dynamic programming. It demonstrates that a truth-informed planner can significantly improve mean coverage over naive approaches, and that reallocating error budgets further enhances coverage across diverse applications such as LLM tool‑calling, content moderation, lesion classification, and recommendation. The study highlights the importance of planning and finite‑sample estimation in ensuring reliable, granular deployment of selective predictors.
By Parivesh Priye, Yufeng Wang, Haibin Ling, Michael Chaykowsky
The paper proposes an ensemble method for clusterwise regression that uses exact solutions on many small random subsamples. Each subsample is solved to global optimality, extended to the full data via nearest-surface assignment, and the resulting partitions are combined by voting or selection. The method achieves high accuracy even with up to 20% gross outliers and can estimate the trimming level without prior knowledge, outperforming traditional trimmed alternation in worst‑case scenarios.
By Samir Orujov
arXiv:2606. 15153v1 Announce Type: new Abstract: Selective prediction with distribution-free risk control promises that, with confidence 1-delta over the calibration draw, the error rate of accepted inputs stays below a user budget alpha.
By Jingwen Zhou, Mingzhe Wang
arXiv:2610.09208v1 Announce Type: cross
Abstract: Weighted conformal prediction corrects for covariate shift by reweighting calibration scores with the likelihood ratio between target and source cova...
By Mehrdad Pournaderi
The paper proposes a framework for studying how alignment difficulty scales with model size, treating alignment as a family of measurable scaling relations for different risk categories. It introduces operational definitions of burden, distinguishes observed, audited, and true alignment, and proves that the largest exponent among corrected risks determines long‑run behavior. The authors preregistered and applied the protocol to public data, finding scaling exponents for attack success, truthfulness, dispositions, sycophancy, and a backdoor, and released interactive browser games to illustrate the laws.
By Jeremy Canale