The paper investigates prompt injection attacks on medical multi‑agent systems that use large language models (LLMs) for clinical decision support. Across two systems, two model families, and three benchmarks, a single injected passage can cause incorrect decisions in up to 61% of cases, and existing defenses only partially mitigate the threat. The authors propose a training‑free defense that verifies each clinical judgment against the patient’s findings, which reduces attack success on MedQA to 4‑7% while preserving accuracy on non‑attacked cases.
arXiv:2606. 14149v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly deployed in healthcare settings, yet their tendency to hallucinate poses risks when clinical decisions are involved.
The paper introduces MIMIC-DOS, a dataset derived from MIMIC-IV that focuses on ICU cases where patient symptoms and medical signs are discordant. It presents CARE, a privacy‑compliant multi‑stage agentic reasoning framework that uses a proprietary LLM to generate structured categories and transitions, while a local LLM performs evidence acquisition and decision‑making. In retrospective evaluations on MIMIC‑DOS, CARE outperforms other LLMs and agentic workflows, demonstrating stronger handling of conflicting clinical evidence while preserving patient privacy.
arXiv:2608. 04053v1 Announce Type: cross Abstract: Prompt injection remains a critical threat to LLM agents, yet existing defenses treat each task as a self-contained problem, independent of previous encounters.
arXiv:2607. 25485v1 Announce Type: new Abstract: Health AI is evolving from answering questions to agentic systems that converse with patients, reason about health records, and act on their behalf.
arXiv:2610.34878v1 Announce Type: new Abstract: Disease diagnosis is a key application of large language model (LLM) agents in healthcare. However, existing medical agents are primarily trained and e...
arXiv:2602.06268v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) systems are increasingly integrated into clinical workflows. However, p...
The paper investigates how human interventions at specific fault points—moments when an AI agent’s reasoning is most vulnerable—affect the diagnostic accuracy of multi‑agent medical systems. Using the MedQA dataset, the authors found that correct interventions can boost baseline accuracy by up to 40%, whereas incorrect or bias‑related interventions can reduce performance by up to 6% and increase diagnostic drift and uncertainty. The study also highlights behavioral parallels between cognitive biases observed in simulated agent conversations and real‑world clinical practice, such as premature closure and susceptibility to misleading cues.
arXiv:2601. 16529v4 Announce Type: replace Abstract: Large language models (LLMs) deployed in clinical decision support may acquiesce to patient requests for care that conflicts with evidence-based guidelines.
The paper introduces DEFER1, a deterministic-first enforcement system for large‑language‑model based multi‑agent systems that uses 28 checks to block most attacks and refers only a small fraction to human judges. In tests across four domains, DEFER1 reduces attack success from about 30% to roughly 3%, with 78% of attacks blocked deterministically and only a quarter reaching the judges. The study highlights that rules effectively handle clear policy violations while judges address ambiguous intent, but also reveals weaknesses such as a risk‑score gate that misclassifies many proposals.
arXiv:2608. 11420v1 Announce Type: new Abstract: Medical diagnostic reasoning is a high-impact use case for LLMs that carries significant implications for the health and wellbeing of users.
arXiv:2606. 16149v3 Announce Type: replace Abstract: Rare disease diagnosis depends on expert reasoning that is scarce and difficult to transfer; off-the-shelf large language models (LLMs) rank the correct disease first in only 35.
Disease diagnosis is a key application of large language model (LLM) agents in healthcare. However, existing medical agents are primarily trained and evaluated with static question-answering data, pro...