arXiv AI June 2, 2026 By Rohith Reddy Bellibatlu, Manpreet Singh, Yash Jajoo, Shyamal Lakhanpal, Abhishek Israni

RISED: A Pre-Deployment Evaluation Framework for High-Stakes AI Decision-Support Systems, with Application to Healthcare

arXiv:2605. 12895v2 Announce Type: replace-cross Abstract: Clinical decision-support systems are expert systems whose recommendations clinicians act on directly, yet they are usually cleared on one aggregate accuracy number from a held-out test set.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv Machine Learning

Jul 31

KAISEN: Reproducible Subgroup Fairness Auditing for Clinical Risk Models

arXiv:2607. 28608v1 Announce Type: new Abstract: Clinical risk models routinely achieve strong aggregate performance while producing materially different error rates across patient subgroups.

By Sparsh Roy, Samuel Girmachew, Nishita Chavan

benchmarks safety

arXiv AI

Jun 3

AI Rater Discrimination Depends on Scoring Protocol in Complex Clinical Decision-Making

arXiv:2606. 03198v1 Announce Type: cross Abstract: Clinical AI evaluation increasingly delegates scoring to large language models (LLMs) acting as AI raters, yet their scoring behavior across evaluation conditions has not been quantitatively characterized.

By Sangwon Baek, Kyu Yeon Hur, Kyunga Kim

llms

arXiv AI

Jun 8

Multi-Agent Reasoning with Consistency Verification Improves Uncertainty Calibration in Medical MCQA

arXiv:2603. 24481v2 Announce Type: replace Abstract: Miscalibrated confidence scores are a practical obstacle to deploying AI in clinical settings.

By John Ray B. Martinez

llms agents nlp benchmarks

arXiv AI

Jul 22

Evaluating medical AI under missing information: same-provider judges and human raters change apparent safety

arXiv:2607. 18828v1 Announce Type: new Abstract: Readiness stress-testing of medical AI has focused on closed-ended and multimodal benchmarks.

By Koyar Afrasyab

llms multimodal benchmarks safety

arXiv AI

Jul 17

MedFailBench: A Clinician-Built Open-Source Benchmark for Medical AI Safety Boundary Inspection

arXiv:2607. 15166v1 Announce Type: new Abstract: Most medical AI benchmarks measure whether a model knows the correct answer.

By Goktug Ozkan

benchmarks safety

arXiv AI

Jul 28

Closed-Loop Validation-Repair for Healthcare Interoperability: A Multi-Model Study of Schema Compliance in Clinical LLMs

arXiv:2607. 24371v1 Announce Type: cross Abstract: Healthcare interoperability requires AI systems to produce structured outputs conforming to standardized schemas including ICD-10 for diagnostic coding, CPT for procedure billing, and HL7 FHIR for data exchange.

By Jianru Shen

llms

Related stories

KAISEN: Reproducible Subgroup Fairness Auditing for Clinical Risk Models

AI Rater Discrimination Depends on Scoring Protocol in Complex Clinical Decision-Making

Multi-Agent Reasoning with Consistency Verification Improves Uncertainty Calibration in Medical MCQA

Evaluating medical AI under missing information: same-provider judges and human raters change apparent safety

MedFailBench: A Clinician-Built Open-Source Benchmark for Medical AI Safety Boundary Inspection

Closed-Loop Validation-Repair for Healthcare Interoperability: A Multi-Model Study of Schema Compliance in Clinical LLMs