MIRA is a bilingual benchmark that evaluates whether large language models (LLMs) provide consistent medical information across different user phrasings, languages, and health literacy levels. It contains 4,320 prompts derived from 60 medically reviewed low‑risk health questions and reveals that models tend to omit key information and offer fewer concrete next steps when responding to low health‑literacy signals, a phenomenon termed Differential Information Dilution (DID). A knowledge‑guided mitigation prompt can reduce this dilution for most models, notably improving Claude and Qwen.
arXiv:2605. 18937v2 Announce Type: replace Abstract: Patient-managed Personal Health Records (PHRs) promises to empower patients to better understand their health; but information in the record is complex, potentially hindering insights.
arXiv:2611.00460v1 Announce Type: cross Abstract: Large language models (LLMs) now answer medical exam questions at or above expert level, but it is unclear whether such scores reflect clinical compe...
arXiv:2604.26766v2 Announce Type: replace-cross Abstract: Accurate and consistent Emergency Severity Index (ESI) assignment remains a persistent challenge in emergency departments, where highly varia...
The paper introduces Expected‑Severity‑Risk (ESR), a new objective for selecting questions in proactive medical dialogue that prioritizes reducing the expected severity of diagnostic errors rather than merely uncertainty. ESR uses population statistics to marginalize over possible answers and distills its rankings into a prefix‑only language policy, enabling deployment without teacher‑side risk computation. Experiments on DDxPlus show ESR cuts high‑severity diagnostic misses by 29.5% and boosts accuracy while adding only 0.14 extra questions per dialogue.
arXiv:2508. 00923v3 Announce Type: replace Abstract: Large language models (LLMs) are increasingly used to answer health-related questions and support healthcare workflows, yet evidence for their safety still relies heavily on static benchmarks that can rapidly become obsolete or be optimized against.
arXiv:2608. 09080v1 Announce Type: cross Abstract: Large Language Models (LLMs) have achieved strong performance in medical question answering and clinical reasoning tasks.
arXiv:2507. 02983v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) hold significant promise for transforming digital health by enabling automated medical question answering.
arXiv:2606. 07951v1 Announce Type: cross Abstract: Humans increasingly turn to Language Models (LMs) in ways that shape beliefs and drive decisions, including discussing, rewriting, and summarizing information from scientific articles, news, and medical reports.
arXiv:2606. 07237v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly used in healthcare for tasks such as clinical question answering, diagnosis support, and report summarization.
arXiv:2601.09717v2 Announce Type: replace-cross Abstract: Online medical consultations contain sensitive health information whose privacy implications depend not only on the entities mentioned but al...
HerHealthEval is a controlled evaluation framework that tests multilingual understanding of women's-health communication across English, French, and Modern Standard Arabic. It provides six communicative forms—canonical, clinical, layperson, indirect or hedged, emotionally concerned, and deliberately under-specified—each conveying the same clinical concern except the under-specified form omits details to assess clarification needs. The study evaluates multilingual instruction models and QLoRA-adapted variants on tasks such as concern classification, risk calibration, clarification behavior, parse compliance, and cross-form consistency, revealing that high aggregate accuracy can mask safety-relevant failures and that language-invariant risk labels improve performance.
The paper introduces HarmReduction, a benchmark for evaluating large language models (LLMs) on their ability to provide accurate and safe harm reduction information to people who use drugs (PWUD). The benchmark, HR-Basic, contains 2,160 question‑answer‑evidence pairs covering safety boundary checks, quantitative value provision, and polysubstance risk inference. Experiments show that even state‑of‑the‑art LLMs struggle with accuracy and can pose severe safety risks, underscoring the need for a dedicated evaluation framework.