arXiv AI

The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing

The paper discusses how autonomous AI systems are moving from advisory to agentic roles in medication prescribing, citing recent U.S. legislation and a Utah pilot program. It argues that three architectural features—calibrated per‑prediction confidence, clear differentiation between epistemic and aleatoric uncertainty, and inferential transparency—are essential for safe autonomous prescribing. A survey of 136 U.S. clinicians shows they require a confidence‑based escalation mechanism, prefer different handling of uncertainty types, and will only accept liability when transparency allows informed decision‑making.

arXiv AI
Aug 5

Optimal Liability Design for Medical AI

arXiv:2608. 03114v1 Announce Type: cross Abstract: Artificial intelligence (AI) is increasingly integrated into medical decision-making, yet its liability implications remain complex, particularly when physicians differ in diagnostic skills and their quality is unobservable.

By Rui Mao, Tingliang Huang, Houcai Shen
arXiv AI
Jul 1

Agentic AI Enhances Physician Trust in Clinical Decision Making

arXiv:2606. 30658v1 Announce Type: cross Abstract: Medical AI has shifted from reasoning to agentic AI, a new paradigm that autonomously invokes external tools during reasoning, rendering intermediate reasoning steps and tool outputs transparent to users.

By Zhiling Yan, Zhe Fang, David J King, Ann Pongsakul, Eashan Adhikarla, Hui Ren, Sunyang Fu, Quanzheng Li, Lifang He, Xiang Li, Hongfang Liu, Yonghui Wu, Lichao Sun
arXiv AI
Aug 17

Algorithm Design and Physician Liability

arXiv:2608. 13618v1 Announce Type: new Abstract: A single clinical algorithm can deliver unequal accuracy across patient groups, and concern about such disparity has grown as artificial intelligence (AI) spreads through clinical decision-making.

By Shujie Luan, Shubhranshu Singh, Tinglong Dai
arXiv AI
Jun 2

Algorithmic Authority and the Clinical Standard of Care

arXiv:2606. 00044v1 Announce Type: cross Abstract: The integration of artificial intelligence into clinical medicine creates a fundamental tension between algorithmic probabilistic reasoning and the experiential intuition of expert physicians; applying Lawrence Lessig's \enquote{Code is Law} framework, I argue that the architecture of clinical AI systems already functions as de facto medical regulation, reshaping liability and the standard of care.

By Aizierjiang Aiersilan
arXiv AI
Aug 3

Reasoning in Real World Clinical Care: Why Large Language Models Are Not Yet Safe for Autonomous Clinical Decision Support

arXiv:2607. 28677v1 Announce Type: new Abstract: LLM now pass medical licensing examinations and, in curated cases, can rival physicians at diagnostic reasoning.

By Shayndhan Sivanathan, Shravan Nageswaran, Mehdi Zadem, Ryaan Sultan, Nicolas von Mallinckrodt, Max Solovyev, Alexey Matyushkin, Sumon Sadhu, Gabriele C DeLuca, Sanjeeva Jeyaretna, James Hillis, Manoj Ramachandran, Prakash Jayakumar
arXiv AI
Jul 9

SycoEval-EM: Sycophancy Evaluation of Large Language Models in Simulated Clinical Encounters for Emergency Care

arXiv:2601. 16529v4 Announce Type: replace Abstract: Large language models (LLMs) deployed in clinical decision support may acquiesce to patient requests for care that conflicts with evidence-based guidelines.

By Dongshen Peng, Yi Wang, Austin Schoeffler, Sun-ha Hong, Brian Suffoletto, David Kim, Carl Preiksaitis, Christian Rose
arXiv AI
Sep 2

AI Morbidity and Mortality: A Framework for Clinical AI Failure Review

AI Morbidity and Mortality (AI M&M) is a structured, blameless framework designed to review clinical AI failures. It combines standardized case intake, evidence preservation, investigator reconstruction, tool‑in‑loop attribution, and corrective‑action tracking, classifying each event across four linked dimensions: Trigger, Mechanism, Clinical Pathway, and Corrective Action. The authors demonstrate the framework with five outpatient medication and clinical decision‑support cases, achieving full agreement among reviewers on all classification axes.

By Paulius Mui, Dean F. Sittig, Steve Labkoff, Sanjay Basu
arXiv Computation and Language
Sep 4

Accountable AI with Grounded, Faithful, Consistent, Actionable Rationales: A Case Study in Clinical Trial Matching with VERDICT

The paper introduces VERDICT, an LLM-based agent that converts clinical trial matching tasks into SMT problems to ensure consistent policy application and accountable decisions. VERDICT outperforms other LLM-only and neurosymbolic baselines on accuracy, achieves perfect policy consistency, and generates clinician-preferred rationales grounded in explicit assumptions and pivotal conditions. It also demonstrates improved counterfactual self‑faithfulness, meaning changes in pivotal conditions appropriately alter decisions.

By Zikai Zhou, Yufei Jin, Yilin Xu, Yu-Chiang Wang, Chieh-Ju Chao, Monica S. Lam