The paper introduces RADAR, a framework for repairing and auditing confidence scores in large language models (LLMs). It separates confidence repair from deployment decisions and from error auditing, proposing Probe‑Anchored Isotonic Calibration (PAIC) for monotone mapping and a prompt‑clustered kernel test for detecting residual errors. Experiments across checkpoints and benchmarks show that richer predictors can improve confidence estimation, but projecting back to a scalar score can lose gains, and lower prediction risk does not always lead to better review decisions.
By Arijit Dey, Bhartendu Kumar, Ramya Hebbalaguppe, Khuushi Maheshwari
arXiv:2607. 18278v1 Announce Type: cross Abstract: Calibration is usually evaluated in aggregate, but the most dangerous failures are often local: predictions that remain highly confident despite being wrong.
By Filippo Cenacchi, Longbing Cao, Runze Yang
arXiv:2606. 21875v2 Announce Type: replace-cross Abstract: Modern data analysis usually gives a prediction without showing whether the evidence behind it is clear, conflicting, or stable.
By Jeffery Opoku, David Banahene
The paper introduces a distribution‑free certification layer that can be applied to any crash‑severity prediction model without modifying the model itself. It provides guarantees for ordinal outcomes, per‑class validity, transfer of coverage to unobserved severities, and one‑sided certificates under deployment shift, all grounded in a functional of the true data law. The framework is evaluated on 5.2 million Texas records, demonstrating a model‑independent lower bound on set width for vulnerable road users and is released as an open‑source package with theorem‑level tests.
By Amir Rafe, Subasish Das
arXiv:2608. 11994v1 Announce Type: new Abstract: We propose claim-level falsification as a principle for test-time scaling and instantiate it through Claim-Level Reliability Assessment (CLR), a training-free framework that reallocates test-time compute from additional solution sampling to targeted verification.
By Sen Xu, Wei Wang, Shixi Liu, Jixin Min, Yingwei Dai, Zhibin Yin, Yirong Chen, Junlin Zhang
arXiv:2608. 08126v1 Announce Type: new Abstract: Credit scoring increasingly relies on models whose decision logic cannot be read off their parameters, in tension with supervisory expectations that adverse decisions be explainable.
By Gregorius Reynaldi Pratama, Kuo-Kun Tseng
arXiv:2607. 18240v1 Announce Type: new Abstract: Large language models (LLMs) can achieve strong fact-checking accuracy, yet forced binary decisions conceal a critical reliability problem: systems may issue confident verdicts even when supporting evidence is weak, sparse, or internally inconsistent.
By Dekun Yang
arXiv:2610.20417v1 Announce Type: new
Abstract: Audits of large language model (LLM) extraction from full-length documents report severe unreliability, leaving two questions open: what the failures a...
By Jiahong Qian
The paper introduces a framework for evaluating AI systems that not only checks final labels but also tracks the reasoning behind them through three core sources—grounds, norms, and authority—forming an eight-cell counterfactual judgment cube. It defines minimal source replacement sets, called judgment receipts, to explain changes in verdicts and provides certification cost bounds for black-box evaluators. The authors present ReasonBench, a benchmark with 19,520 cases, and demonstrate that while high standard accuracy can mask robustness issues, receipt accuracy reveals significant gaps in reasoning consistency across different models.
By Ye Chen, Weining Zhang
arXiv:2608. 10406v1 Announce Type: cross Abstract: Web search, product search, and question-answering retrieval systems often assign a relevance label and confidence score to each query-candidate pair.
By Inwoo Tae, Yongjae Lee
The paper argues that prediction‑based certifications—such as accuracy, calibration, and conformal coverage—are insufficient to guarantee trustworthy AI. It proves a separation theorem showing that a model can appear reliable under all prediction‑side certificates yet differ arbitrarily in explanation fidelity and deployment behaviour. The authors propose a competence envelope framework that combines both prediction and explanation certification to detect such hidden failures.
By Nataliya Shakhovska, Ivan Izonin, Stergios-Aristoteles Mitoulis
arXiv:2608. 09768v1 Announce Type: new Abstract: A prediction that is both confident and wrong is a critical reliability failure because it can bypass abstention and human review precisely when the model is mistaken.
By Ange-Cl\'ement Akazan, Ineza Remy Mugenga, Abebe Geletu, Jean Medard Ngnotchouye, Issa Karambal