The study investigates how medical AI agents can silently access protected health information (PHI) without revealing it, a practice that bypasses the HIPAA minimum‑necessary standard. By embedding canaries in synthetic patient records, the authors evaluated five language models across multiple boundary settings and attack scenarios, finding that some models accessed PHI unnecessarily while stricter boundaries and pre‑execution gates largely mitigated this behavior. The research demonstrates that output‑only evaluations fail to detect such covert access, underscoring the need for combined boundary controls, gating mechanisms, and access‑log audits.
Medical artificial intelligence agents should retrieve patient identifiers only when a task requires them, the data-minimization principle behind the HIPAA minimum necessary standard for protected hea...
arXiv:2607. 13411v1 Announce Type: cross Abstract: Clinical AI models can expose patients to harm when adversarial vulnerabilities go undetected, yet formal security auditing requires statistical expertise, specialized tools, and significant time.
arXiv:2508. 00923v3 Announce Type: replace Abstract: Large language models (LLMs) are increasingly used to answer health-related questions and support healthcare workflows, yet evidence for their safety still relies heavily on static benchmarks that can rapidly become obsolete or be optimized against.
The paper introduces MIMIC-DOS, a dataset derived from MIMIC-IV that focuses on ICU cases where patient symptoms and medical signs are discordant. It presents CARE, a privacy‑compliant multi‑stage agentic reasoning framework that uses a proprietary LLM to generate structured categories and transitions, while a local LLM performs evidence acquisition and decision‑making. In retrospective evaluations on MIMIC‑DOS, CARE outperforms other LLMs and agentic workflows, demonstrating stronger handling of conflicting clinical evidence while preserving patient privacy.
arXiv:2607. 25485v1 Announce Type: new Abstract: Health AI is evolving from answering questions to agentic systems that converse with patients, reason about health records, and act on their behalf.
The paper introduces a lightweight screening tool that evaluates individual patient vulnerability to membership inference attacks across thousands of medical AI models. By measuring attack scores for each patient, the authors found that certain records consistently rank high in risk regardless of training settings, and that stronger weight decay can reduce overall exposure but not the identity of the most at‑risk patients. The tool uses four small models to flag high‑risk patients before a production model is trained, aiming to integrate privacy assessment into clinical AI data preparation.
arXiv:2606. 14149v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly deployed in healthcare settings, yet their tendency to hallucinate poses risks when clinical decisions are involved.
The paper investigates how large language model agents can inadvertently leak confidential information by combining data from multiple documents—a phenomenon termed the "mosaic effect." Using 30 synthetic cybersecurity, medical, and military scenarios, the authors generate 10,800 simulated and 1,440 live answers from three LLM agents, finding that even the best model discloses forbidden composites in over 7% of simulated responses. While visible classification headers and aggregation rules reduce disclosure rates, unsafe disclosures remain in roughly one third of simulated answers, and increased refusals do not fully prevent leakage of prohibited conclusions.
Synthetic Hospital is an open, fully synthetic longitudinal electronic health record benchmark created from public medical education material. It contains 1,268 patients and 5,602 encounters, with every diagnosis, finding, and temporal relation grounded in standard ontologies and traceable back to its source. Physicians could not reliably distinguish its records from real charts, and current frontier AI models perform significantly below human experts on tasks such as reconstructing problem lists and summarizing charts.
The article discusses how Clinical AI can predict outcomes based on data generated by existing clinical judgment, but such performance alone does not prove usefulness before recognition or across care processes. It proposes evidence requirements tailored to the intended use and information conditions, including state‑reference and proxy‑relative timing tests, to strengthen recognition claims. The framework aims to help hospitals determine whether the conditions that support performance are present when assistance is most needed, making questions that current evaluation practices overlook explicit and specifying the direct evidence required to answer them.
MedDeID is an on‑premises framework that combines in‑house annotation, synthetic‑note generation, model training, inference, pseudonymisation and evaluation to de‑identify clinical text. On a Dutch hospital benchmark, a hospital‑trained transformer detected 98.9 % of identifying text while redacting only 0.24 % of non‑identifier text; a synthetic‑only model achieved 96.1 %. In primary‑care notes, the synthetic‑trained model outperformed the hospital‑trained model in recall and robustness to identifier‑format changes, and an English version trained without real text reached 99.7 % and 98.9 % detection on synthetic benchmarks.
arXiv:2512. 01241v3 Announce Type: replace-cross Abstract: Large language models (LLMs) are routinely used by physicians and patients for medical advice, yet their clinical safety profiles remain poorly characterized.