arXiv AI
LWCal: Loss-Weighted Calibration for Tabular Classifiers with Noisy Calibration Labels
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jun 2
CalArena: A Large-Scale Post-Hoc Calibration Benchmark
arXiv:2605. 30188v2 Announce Type: replace-cross Abstract: Reliable probability estimates are critical in many machine learning applications, yet modern classifiers are often poorly calibrated.
arXiv Machine Learning
Jun 2
Decision-Path Patterns as Tree Reliability Signals: Path-based Adaptive Weighting for Random Forest Classification
arXiv:2605. 20716v5 Announce Type: replace Abstract: Random forests construct each tree with a different, randomised representation of the feature space.
arXiv Machine Learning
Jul 16
Temperature Scaling Is Not Enough: Calibration Gaps Under Human Label Distributions
arXiv:2607. 13423v1 Announce Type: new Abstract: Temperature scaling is the dominant post-hoc calibration method in modern deep learning.
arXiv Machine Learning
Aug 5
Evaluation Blindness: How Silent Measurement Failures Corrupt AI Systems from Training to Deployment
arXiv:2608. 02786v1 Announce Type: new Abstract: AI systems can fail silently.
arXiv Machine Learning
Sep 14
Can We Trust LLM Judges: A Study of Capability-Dependent Biases and Multi-Judge Ensemble for Bias Calibration
The paper investigates how large language models (LLMs) used as judges in absolute scoring tasks exhibit systematic biases that compromise reliability. It shows that a judge’s task accuracy strongly predicts both its judging accuracy and its directional bias, yet more capable examinee models consistently receive more lenient judgments. To mitigate these biases, the authors propose a calibrated weighted majority voting (WMV) ensemble that estimates judges’ error rates from inter-judge agreement patterns, achieving near-oracle performance without labeled data and improving both accuracy and fairness.
arXiv Machine Learning
Jul 21
The Calibration Channel Determines the Bayes-Error Proxy: An Exact Law for Temperature-Induced Distortion
arXiv:2607. 18162v1 Announce Type: new Abstract: The soft-label Bayes-error estimator beta(z) = E[min(z, 1-z)] of Ishida et al.