The study evaluates how clinical language models perform when patient narratives are presented in different voices—clinical, socioeconomic, and cultural—using 1,000 MedQA cases paired with three retellings each. All 12 tested models show a significant drop in accuracy (about 26% more errors) when the patient speaks in a marked voice, a penalty that persists even when the original vignette is included verbatim and is not explained by measured retelling properties. The authors find that rewriting the patient’s account into a clinical vignette before answering can halve this accuracy gap, though structured schema pipelines and additional reasoning tools offer limited improvements.
arXiv:2608. 15254v1 Announce Type: new Abstract: Clinical-AI guidance increasingly recommends prompting language models to reason with attention to diversity, equity, and inclusion (DEI).
arXiv:2603.11413v4 Announce Type: replace-cross Abstract: A recent Nature Medicine study reported that ChatGPT Health under-triages 51.6% of emergencies and concluded that consumer-facing AI triage p...
The paper investigates how medical large language models (LLMs) may exhibit narrative anchoring bias when presented with the same clinical case in different patient voices. Using the NarrativeShield SDoH MedQA dataset, the authors evaluate three Qwen2.5 instruction‑tuned LLMs (1.5B, 3B, 7B) on 300 clinical cases, reporting metrics such as persona‑level accuracy, counterfactual consistency, correct consistency, and narrative sensitivity error. The 7B model achieves the highest accuracy (56.33 %) and correct consistency (40.33 %), yet narrative sensitivity errors remain substantial (31.67 %).
arXiv:2610.08585v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly relied upon to support ambient documentation and clinical reasoning. Here we examine the impact of a...
The paper investigates whether clinical concepts are represented as distinct, causally influential centers within the latent space of large language models (LLMs). By evaluating eleven open-weight LLMs, the authors discover that each model contains dedicated clinical concept centers that are interpretable, activate only on relevant clinical narratives, and drive model behavior in both constrained and open-ended contexts. These centers can be leveraged for evaluation and performance improvement, as steering models along them enhances downstream clinical outcomes and aligns with clinician preferences.
arXiv:2606. 05970v1 Announce Type: cross Abstract: Large language models are increasingly used for structured extraction from clinical free-text notes, but the sensitivity of their output to upstream configuration choices is less understood than their accuracy on fixed benchmarks.
Large language models (LLMs) are increasingly relied upon to support ambient documentation and clinical reasoning. Here we examine the impact of a failure mode shared between these two applications by...
IatroBench is a pre‑registered benchmark that evaluates language models on clinical omission and commission harms across 60 scenarios and six models. Using a physician‑written rubric scored by Claude Opus 4.6, the study finds that models tend to withhold more information from patients than from doctors—a phenomenon termed framing‑contingent withholding—while also revealing varied patterns of omission across different models. The benchmark highlights how framing influences the amount of medical information shared by AI systems.
arXiv:2610.08585v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly relied upon to support ambient documentation and clinical reasoning. Here we examine the impact of a fail...
arXiv:2604. 17359v2 Announce Type: replace-cross Abstract: Language models asked to simulate psychiatric patients produce cases that survive inspection one at a time and populations that match no real one.
arXiv:2608.31016v1 Announce Type: cross Abstract: Ambient AI scribes draft clinical notes, and published audits find their dominant error is omission: information the encounter established that the n...
arXiv:2503. 10647v2 Announce Type: replace-cross Abstract: This study evaluated the diagnostic reliability of two Large Language Models (LLMs), Google Gemini 2.