arXiv:2607. 28677v1 Announce Type: new Abstract: LLM now pass medical licensing examinations and, in curated cases, can rival physicians at diagnostic reasoning.
By Shayndhan Sivanathan, Shravan Nageswaran, Mehdi Zadem, Ryaan Sultan, Nicolas von Mallinckrodt, Max Solovyev, Alexey Matyushkin, Sumon Sadhu, Gabriele C DeLuca, Sanjeeva Jeyaretna, James Hillis, Manoj Ramachandran, Prakash Jayakumar
arXiv:2607. 13036v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used for decision support in healthcare, but clinical evidence is often incomplete or evolving.
By Oriana Presacan, Andreea Grama, Larisa Irimin\u{a}, Alireza Nik, Jaya Ojha, Vajira Thambawita, Ciprian I. B\u{a}cil\u{a}, Bogdan Ionescu, Michael A. Riegler
The study introduces Safety-Focused Evaluation of Scaling (SaFE-Scale) and applies it to RadSaFE-200, a 200-question radiology benchmark with clinician-defined evidence and safety labels. Thirty-four local large language models were tested under six deployment conditions, showing that providing clean clinician-written evidence significantly improved accuracy and reduced high-risk errors and dangerous overconfidence. The results indicate that safety in clinical LLMs depends more on deployment choices than on model size or inference-time compute.
By Sebastian Wind, Tri-Thien Nguyen, Jeta Sopa, Mahshad Lotfinia, Sebastian Bickelhaup, Michael Uder, Harald K\"ostler, Gerhard Wellein, Sven Nebelung, Daniel Truhn, Andreas Maier, Soroosh Tayebi Arasteh
arXiv:2608.29241v1 Announce Type: new
Abstract: Clinical voice agents are now deployed in routine care, where real patients do not wait their turn: they interrupt. These systems typically use a casca...
By Zachary Ellis, Spencer Hazel, Adam Brandt, Yajie Vera He, Ernest Lim, Jared Joselowitz
The paper introduces ALLMTS, an automated medical triage system that uses large language models (LLMs) to execute commercially maintained telehealth guidelines rather than directly triage callers. By selecting the appropriate guideline and evaluating its triage questions, ALLMTS deterministically derives dispositions, achieving 0.95 accuracy on 566 synthetic scenarios and 0.98 on a 50-scenario benchmark, matching expert nurse performance. The system dramatically reduces over‑triage rates (to ≤0.02) while keeping under‑triage low, demonstrating the effectiveness of coupling LLMs with executable clinical guidelines.
By Micha{\l} Remigiusz Janiszewski, Artur Stopa, {\L}ukasz Lenkiewicz, Wojciech Achtelik, Bartosz Gonczarek
arXiv:2604.26766v2 Announce Type: replace-cross
Abstract: Accurate and consistent Emergency Severity Index (ESI) assignment remains a persistent challenge in emergency departments, where highly varia...
By Manar Aljohani, Brandon Ho, Kenneth McKinley, Dennis Ren, Xuan Wang
The study evaluates counterfactual bias in ten open‑source large language models (LLMs) for pediatric Emergency Severity Index (ESI) prediction. By creating paired clinical vignettes that differ only in demographic or socioeconomic variables, the authors measure shifts in acuity assignment, finding that counterfactual sensitivity varies widely across model families and sizes. A fine‑tuned Qwen2.5‑7B model exhibited the lowest sensitivity, while larger or medical‑domain models sometimes showed greater shifts, highlighting the need for fairness assessment before clinical deployment.
By Manar Aljohani, Brandon Ho, Kenneth McKinley, Dennis Ren, Xuan Wang
Large language models (LLMs) are increasingly used for clinical reasoning, yet their ability to revise judgments as patient evidence changes is uncertain. In this study, researchers evaluated longitudinal belief updating using real intensive‑care patient trajectories and found that conditioning on prior judgments often increased prediction error. Controlled experiments revealed two failure modes: models were more responsive to worsening than improving respiratory evidence, and they shifted estimates significantly when prior risk was altered, indicating a causal influence of prior beliefs. Prompting did not improve reliability, and an Evidence‑Validated Longitudinal Update (EVLU) approach produced fewer but more trustworthy revisions, highlighting a reliability‑coverage trade‑off.
By Min Zeng, Rui Zhang
The paper introduces a benchmark of 400 synthetic healthcare workflow episodes to evaluate large language model (LLM) agents’ reliability in escalating care when evidence is missing or conflicting. It finds that while clean decision accuracy is high (0.97), models miss a large proportion of necessary escalations (32–97%) under uncertainty, and even the largest 9.7B model’s performance drops to 0.78 in conflict scenarios. A workflow-aware controller improves utility and reduces missed escalations, but its effectiveness is limited by lexical generalization failures.
By Zihao Zheng (Washington University in St. Louis), Junyi Yao (Washington University in St. Louis), Baichuan Li (Southern Methodist University), Jiayu Long (Washington University in St. Louis)
arXiv:2609.38480v1 Announce Type: cross
Abstract: Most clinical benchmarks evaluate language models (LMs) on diagnosis using complete case descriptions. In clinical practice, however, patients presen...
By Xueting Fang, Zehui Li, Yang Yang, Camilla Giovino, Shubh K. Patel, Shailly Prajapati, Vallijah Subasri, Caihua Shan
arXiv:2608. 10258v1 Announce Type: cross Abstract: Large language models (LLMs) increasingly provide conversational health information that may influence treatment decisions, yet existing benchmarks do not isolate whether medication-safety boundaries persist across follow-ups after explicit self-treatment intent.
By Waleed Jamil, Raphael Schmitt
The paper introduces DistressBench, a benchmark comprising 718 English clinician-authored conversations covering 23 self-harm subcategories, each evaluated against a weighted rubric across seven crisis-support dimensions. Models are scored by the fraction of criteria they satisfy, revealing that the best models meet 88.4% of weighted criteria while clinicians achieve 98.7%. The study highlights gaps in crisis response, notably that 35.3% of conversations where crisis is recognized fail to route users to human help, underscoring the need to evaluate models on supportive care rather than mere refusal.
By Drew Rein, Patrick Oathout, Vishal Kumar, Udari Madhushani Sehwag