arXiv Machine Learning June 5, 2026 By David Gringras

IatroBench: Pre-Registered Evidence of Iatrogenic Harm from AI Safety Measures

Read the original on arXiv Machine Learning →

arXiv:2604. 07709v4 Announce Type: replace-cross Abstract: A heavily safety-trained model will hand a physician the full, patient-followable benzodiazepine taper and refuse it to the patient who needs it, over identical clinical facts; the knowledge is present either way.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv AI

Jul 22

Evaluating medical AI under missing information: same-provider judges and human raters change apparent safety

arXiv:2607. 18828v1 Announce Type: new Abstract: Readiness stress-testing of medical AI has focused on closed-ended and multimodal benchmarks.

By Koyar Afrasyab

llms multimodal benchmarks safety

arXiv AI

Jul 21

Judge-dependent safety gains and model-specific helpfulness costs of evidence-sufficiency prompting in clinical LLMs

arXiv:2607. 18086v1 Announce Type: new Abstract: Background: LLM judges increasingly score whether clinical language models give overconfident answers under incomplete evidence, yet whether a measured "safety gain" reflects real behavior change or the judge's calibration is unresolved.

By Koyar Afrasyab

llms benchmarks safety

arXiv AI

Aug 3

Reasoning in Real World Clinical Care: Why Large Language Models Are Not Yet Safe for Autonomous Clinical Decision Support

arXiv:2607. 28677v1 Announce Type: new Abstract: LLM now pass medical licensing examinations and, in curated cases, can rival physicians at diagnostic reasoning.

By Shayndhan Sivanathan, Shravan Nageswaran, Mehdi Zadem, Ryaan Sultan, Nicolas von Mallinckrodt, Max Solovyev, Alexey Matyushkin, Sumon Sadhu, Gabriele C DeLuca, Sanjeeva Jeyaretna, James Hillis, Manoj Ramachandran, Prakash Jayakumar

llms agents safety

arXiv AI

Jun 15

Can LLMs Accurately Score Medical Diagnoses and Clinical Reasoning?

arXiv:2604. 14892v3 Announce Type: replace-cross Abstract: Evaluating medical AI systems using expert clinician panels is costly and slow, motivating the use of large language models (LLMs) as alternative adjudicators.

By Amy Rouillard, Sitwala Mundia, Linda Camara, Ziyaad Dangor, Michael Cameron Gramanie, Ismail Kalla, Shabir A. Madhi, Kajal Morar, Marlvin T. Ncube, Haroon Saloojee, Bruce A. Bassett

llms agents benchmarks safety

arXiv AI

Aug 5

Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning

arXiv:2608. 03028v1 Announce Type: new Abstract: Applying a valid medication-safety rule when its patient-specific conditions are not met can produce an incorrect decision.

By Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang

llms benchmarks safety

arXiv Machine Learning

Jul 31

KAISEN: Reproducible Subgroup Fairness Auditing for Clinical Risk Models

arXiv:2607. 28608v1 Announce Type: new Abstract: Clinical risk models routinely achieve strong aggregate performance while producing materially different error rates across patient subgroups.

By Sparsh Roy, Samuel Girmachew, Nishita Chavan

benchmarks safety

Related stories

Evaluating medical AI under missing information: same-provider judges and human raters change apparent safety

Judge-dependent safety gains and model-specific helpfulness costs of evidence-sufficiency prompting in clinical LLMs

Reasoning in Real World Clinical Care: Why Large Language Models Are Not Yet Safe for Autonomous Clinical Decision Support

Can LLMs Accurately Score Medical Diagnoses and Clinical Reasoning?

Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning

KAISEN: Reproducible Subgroup Fairness Auditing for Clinical Risk Models