The study evaluates counterfactual bias in ten open‑source large language models (LLMs) for pediatric Emergency Severity Index (ESI) prediction. By creating paired clinical vignettes that differ only in demographic or socioeconomic variables, the authors measure shifts in acuity assignment, finding that counterfactual sensitivity varies widely across model families and sizes. A fine‑tuned Qwen2.5‑7B model exhibited the lowest sensitivity, while larger or medical‑domain models sometimes showed greater shifts, highlighting the need for fairness assessment before clinical deployment.
By Manar Aljohani, Brandon Ho, Kenneth McKinley, Dennis Ren, Xuan Wang
arXiv:2608.29582v1 Announce Type: cross
Abstract: Current evaluations of large language models (LLMs) primarily focus on factual knowledge retrieval, overlooking the fundamental challenge of navigati...
By Yi Yu, Bo Wang, Chong Feng, Ge Shi, Xia Liu, Ziyi Yang, Xuewen Shi
arXiv:2606. 05970v1 Announce Type: cross Abstract: Large language models are increasingly used for structured extraction from clinical free-text notes, but the sensitivity of their output to upstream configuration choices is less understood than their accuracy on fixed benchmarks.
By Martin Murin
Background: Disease severity is a multidimensional construct difficult to capture with rule-based approaches in Electronic Healthcare Records (EHR). Agentic large language model (LLM) systems could synthesise clinical evidence and reason over EHRs, but remain unevaluated for this task.
The paper introduces ClinGeX, a large‑language‑model agent that uses 105 curated executable skills to automate clinical genomic analysis. In benchmark tests, ClinGeX achieved 71.7% macro‑averaged task‑level concordance, outperforming the best comparator at 45.7%. Human‑AI collaboration studies showed ClinGeX significantly improved case‑level concordance across variant analysis, rare disease interpretation, and oncology decision‑making, while also reducing interpretation time.
By Qiaoyu Zheng, Weike Zhao, Yanjie Fan, Ye Cao, Mengqi Jiang, Xiao Zhou, Xiaoman Zhang, Pengcheng Qiu, Yongguo Yu, Yanfeng Wang, Ya Zhang, Chaoyi Wu, Weidi Xie
The paper introduces VERGE, a verification-enhanced refinement workflow that extracts six red‑flag symptoms and family‑history risk status for early‑onset colorectal cancer from free‑text clinical notes. VERGE uses retrieval‑augmented generation followed by a bounded verification‑refinement cycle that checks textual grounding and clinical validity, correcting claims until resolved or escalating to human review. In evaluation on 4,033 clinician‑labeled note‑finding pairs, VERGE improved precision from 0.764 to 0.849 and MCC from 0.681 to 0.730 compared to a single‑agent baseline, while requiring human review for only 1.5 % of claims.
By Nikkie Hooman, Monarch Nigam, Amy E. Hughes, Rasmi G. Nair, Mehak Gupta