The study compared 13 decision models with 15 generative large language model configurations on 1,273 four‑option MedQA questions. Decision models achieved 67.71%–89.08% accuracy, while generative models ranged from 78.32%–96.86%. Mercury Decide offered the lowest cost per 1,000 questions and Clef Omni the fastest median request time, but decision models still showed higher error rates for high‑confidence predictions and varied probability repeatability.
By Oscar Freyer, Daniel Glauert, Moritz Hofmann, Max Ostermann, Stephen Gilbert, Jakob Nikolas Kather
The paper introduces three interpretable metrics—Explanation Stability Index (ESI), Causal Faithfulness Score (CFS), and Perturbation Stability Score (PSS)—to assess the faithfulness of large language models (LLMs) in clinical decision support. Evaluating six LLMs on 150 MedQA-USMLE questions, the study finds that only 23.3% of cited clinical concepts are causally necessary, and that correct answers often have lower CFS than incorrect ones, with answer consistency negatively predicting faithfulness. These findings suggest that accuracy, consistency, and consensus are incomplete safety signals for clinical decision-making support.
By Bharath Kumar Bolla, Chaitali Deb Purkayastha, Vishnu Surya Reddy Nandi
arXiv:2608. 09080v1 Announce Type: cross Abstract: Large Language Models (LLMs) have achieved strong performance in medical question answering and clinical reasoning tasks.
By Maryam Tahermazandarani, Adnan Mahmood, Fahmida Islam, Quan Z. Sheng
arXiv:2603. 24481v2 Announce Type: replace Abstract: Miscalibrated confidence scores are a practical obstacle to deploying AI in clinical settings.
By John Ray B. Martinez
MedJev is an open System One model that answers medical questions in a single forward pass without generating text, enabling fast, local clinical decision support. Across ten medical benchmarks, MedJev achieves competitive accuracies—72.5 % on MedQA, 75.4 % on USMLE, and 80.1 % on NEJM—while maintaining well‑calibrated confidence estimates. On patient‑level tasks, it attains an AUROC of 0.926 for in‑hospital mortality, surpassing XGBoost, and introduces a lightweight two‑parameter recalibration method that reduces miscalibration from 0.286 to 0.007 using only 50–200 labeled examples.
By Simon A. Lee, Jiahang Sha, Moonseong Jeong, Yunbei Pan, Jeffrey N. Chiang
OmniMed-Jev is a new medical multimodal model that represents each decision as a Choice, Noul, or Score over a runtime-supplied candidate set, returning a full probability distribution for each decision. By unifying diverse imaging modalities and prediction tasks into a single candidate-conditioned probability model, it makes heterogeneous outputs comparable probabilities rather than task-specific strings. In controlled comparisons against a generative baseline, OmniMed-Jev’s reported probabilities align more closely with observed correctness, reducing calibration error by up to an order of magnitude and reliability error by up to two, while maintaining comparable point-prediction performance.
By Luyao Tang, Cheng Chen
The paper introduces Clinical System One (CS1), a 4‑billion‑parameter decision model that rapidly produces typed judgments from clinical records. Trained on 12.4 million Q&A pairs from 3.6 million patients, CS1 can answer multiple questions in a single forward pass, providing probability estimates for uncertain cases. In evaluations on 963,408 held‑out patients and 21 external datasets, CS1 outperformed larger open models and, when paired with a language‑model agent, efficiently identified trial‑eligible patients with far fewer tokens.
By Andrew Zhang, Trinh T. L. Vuong, Sophia J. Wagner, Dongmin Bang, Tong Ding, Yuchang Su, Mary Cipperman, Christina S. Chen, Judith Lefkes, Ramin Kahidi, Ming Y. Lu, Caiwei Tian, Tianhao Luo, Jingnan Ma, Rowland Pettit, Long Phi Le, Faisal Mahmood
arXiv:2507. 02983v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) hold significant promise for transforming digital health by enabling automated medical question answering.
By Mohammad Anas Azeez, Rafiq Ali, Ebad Shabbir, Zohaib Hasan Siddiqui, Gautam Siddharth Kashyap, Jiechao Gao, Usman Naseem
RetiJev is a family of 0.8B‑9B decision models designed for ophthalmic clinical decision support, offering fast, accurate, and privacy‑preserving inference without autoregressive decoding. The authors created RetiJev‑Data, a source‑grounded corpus of 78,158 clinical‑decision and diagnosis questions, 50,074 guideline‑grading questions, and a 24,662‑question evaluation benchmark. RetiJev outperforms a 27B LLM and the commercial Jev 1.13 model on guideline grading and clinical decisions, achieving the highest accuracy while answering a case in 158 ms on a single GPU—3.5× faster than LLM generation and 470× faster than reasoning.
By Bo Liu, Lianyu Wang, Meng Wang, Ching-Yu Cheng, Huazhu Fu
The study evaluates four zero‑shot System 1 models on 86 public clinical tasks across 14 domains and 20 languages, measuring performance via AUROC. The best models achieved high AUROC scores—0.94 for medical knowledge, 0.93 for text understanding, 0.83 for clinical judgement from text, 0.89 for image perception, 0.84 for 2D image findings, and 0.69 for volumetric findings—yet performed worse on imaging tasks and varied significantly with language. System 1 models were fast (0.34–0.44 s) and energy‑efficient (24–81 J) but lagged behind clinicians on triage and were outperformed by generative models in some metrics.
By Xiufeng Yang, Konstantinos Vrettos, Zeineb Ben Chaaben, Fabian Drexel, Jannik L\"ubberstedt, Johannes Moll, Anirudh Narayanan, Frederike Stumpf, Cosmin I. Bercea, Jiawen Wang, Evangelia E. Vassalou, Krischan Braitsch, Sebastian Ziegelmayer, Markus Graf, Marcus R. Makowski, Lisa Adams, Nassir Navab, Michail E. Klontzas, Keno Bressem
arXiv:2610.28522v1 Announce Type: new
Abstract: Decision models support routing, guardrails, and output verification in LLM systems. However, item accuracy alone cannot establish whether models make...
By Xialie Zhuang, Mingcong Lei, Zunhai Su, Ri Liu, Jianjin Li, Peixian Ma, Jia Pan, Ngai Wong, Guanhua Chen, Chuan-Wei Kuo
arXiv:2509. 02594v3 Announce Type: replace-cross Abstract: Evaluating large language models (LLMs) on their ability to generate high-quality, accurate, situationally aware answers to clinical questions requires going beyond conventional benchmarks to assess how these systems behave in complex, high-stakes clinical scenarios.
By Sandhanakrishnan Ravichandran, Shivesh Kumar, Rogerio Corga Da Silva, Miguel Romano, Reinhard Berkels, Michiel van der Heijden, Olivier Fail, Valentine Emmanuel Gnanapragasam