arXiv AI
Future Querying: Can LLMs Serve as Implicit Medical World Models?
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jun 26
A Pipeline for Generating Longitudinal Synthetic Clinical Notes Using Large Language Models
arXiv:2606. 26879v1 Announce Type: new Abstract: Synthetic data is increasingly used to enable the development and evaluation of AI systems in domains where access to real-world data is restricted.
arXiv AI
Oct 7
SAGE: Semantic Anchor-Guided Evolution for Grounded Medical QA Data Synthesis
arXiv:2610.08093v1 Announce Type: cross Abstract: Developing reliable models for clinical tasks, such as Medical Question Answering (QA), is severely constrained by the limited availability of high-q...
Hugging Face Trending Papers
Jun 25
A Pipeline for Generating Longitudinal Synthetic Clinical Notes Using Large Language Models
Synthetic data is increasingly used to enable the development and evaluation of AI systems in domains where access to real-world data is restricted. In healthcare, clinical documentation presents particular challenges due to its sensitivity.
Hugging Face Trending Papers
Sep 24
A Living Benchmark for Information Retrieval from Electronic Health Records
The paper introduces BRIE, a scalable framework that automatically creates question–answer pairs from longitudinal electronic health record notes, validated by nineteen clinicians. It offers a continuously maintainable benchmark for evaluating large language models in clinical settings, addressing limitations of manual, costly, and quickly outdated existing benchmarks. Experiments across nine LLMs and five inference strategies reveal that even state‑of‑the‑art systems often miss clinically important information, especially for synthesis‑heavy queries.
arXiv AI
Sep 25
A Living Benchmark for Information Retrieval from Electronic Health Records
The paper introduces BRIE, a continuously maintainable benchmark for evaluating large language models (LLMs) in electronic health record (EHR) information retrieval. It presents a scalable framework that automatically generates question–answer pairs from longitudinal EHR notes, validated by nineteen clinicians. The benchmark allows assessment of multiple inference strategies and highlights that state‑of‑the‑art LLMs often miss clinically important information, especially when synthesis across documents is required.
arXiv Computation and Language
Aug 25
Clinically Grounded Privacy Evaluation of Medical LMs
The paper introduces a clinically grounded privacy evaluation framework for medical language models, assessing leakage across a spectrum of adversarial access levels—from publicly inferable demographics to leaked note fragments. Using this framework on an LM pretrained on 378,000 clinical notes, the authors find that routine encounter metadata leads to high verbatim memorization and significant recovery of sensitive diagnoses (e.g., AUROC 0.91 for abortion, 0.82 for HIV). They also note that exact-match memorization can overstate disclosure, with 36% of memorized tokens being templated documentation, underscoring the risks of training on longitudinal clinical data and offering a reusable evaluation tool.