arXiv AI August 7, 2026 By Omid Bazgir, Md Nasir, Jacob Hoffman, Yang Yang, Manu Agrawal, Anusua Trivedi, Vinay Rao Dandin, Chris Gibbons, Christine Swisher

Improving the Realism of Synthetic Clinical Benchmarks Under Utility Constraints

Read the original on arXiv AI →

arXiv:2608. 06265v1 Announce Type: new Abstract: Synthetic clinical benchmarks for enterprise AI agents can pass existing utility checks and still remain structurally unrealistic, especially in privacy-sensitive healthcare settings where operational data are hard to access.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Machine Learning

Aug 14

CoMedBench: A Multi-Source Benchmark of Synthetic Medical Data Fidelity and Downstream Utility

arXiv:2608. 12805v1 Announce Type: new Abstract: Access to clinical data is essential for developing reliable healthcare machine learning systems, but direct use of electronic health records is constrained by privacy regulation, institutional review, data-use agreements, and the risk of re-identification.

By Akanta Das, Al Amin Farhad, Mrinmoy Sarkar Anto, David Rehkopf, Ayin Vala, Tanmoy Sarkar Pias

benchmarks safety

arXiv AI

Jul 23

Recovering Clinical Utility Under Differential Privacy: Empirical Validation of Adaptive Federated Aggregation on Heterogeneous Cardiovascular Datasets

arXiv:2607. 19403v1 Announce Type: cross Abstract: Validating federated learning frameworks on real clinical data is an essential step between proof-of-concept demonstrations in controlled synthetic environments and deployment in real multicenter healthcare settings.

By Rodrigo Tertulino, Laercio Alencar, Ricardo Almeida

benchmarks safety

arXiv Machine Learning

Jun 8

Accelerating Reproducible Research in Synthetic EHR Generation

arXiv:2606. 06990v1 Announce Type: new Abstract: The generation of high-fidelity synthetic Electronic Health Records (EHR) is crucial for advancing medical research while preserving patient privacy.

By Jalen Jiang, Chufan Gao, Ethan Rasmussen, Stephen Z. Xie, Jimeng Sun

llms diffusion benchmarks safety

arXiv AI

3d ago

EHR-RobustGym: Benchmarking and Training Agents for Robust Clinical Reasoning

arXiv:2609.39371v1 Announce Type: new Abstract: In hospital workflows, electronic health records (EHRs) are often noisy, and may not contain the evidence needed to confirm events or measurements refe...

By Yitong Qiao, Yancheng Jin, Lei Liu, Yue Shen, Jian Wang, Jinjie Gu, Zhixuan Chu

llms agents reinforcement-learning fine-tuning benchmarks data-engineering

arXiv AI

Sep 25

Synthetic Hospital: An Open, Verifiable, Physician-Validated Longitudinal EHR Benchmark

Synthetic Hospital is an open, fully synthetic longitudinal electronic health record benchmark created from public medical education material. It contains 1,268 patients and 5,602 encounters, with every diagnosis, finding, and temporal relation grounded in standard ontologies and traceable back to its source. Physicians could not reliably distinguish its records from real charts, and current frontier AI models perform significantly below human experts on tasks such as reconstructing problem lists and summarizing charts.

By Christine Park, Valerie Chen, Tim Dettmers

llms benchmarks safety

arXiv AI

Aug 11

FoMoH: A clinically meaningful foundation model evaluation for structured electronic health records

arXiv:2505. 16941v4 Announce Type: replace-cross Abstract: Foundation models (FMs) promise to address core limitations of traditional supervised machine learning: (i) reliance on large amounts of labeled data, (ii) task specificity, and (iii) poor transportability.

By Vincent Jeanselme, Zilin Jing, Aparajita Kashyap, Chao Pang, Florent Pollet, Young Sang Choi, Xinzhuo Jiang, Yuta Kobayashi, Yanwei Li, Sara Matijevic, Karthik Natarajan, Shalmali Joshi

benchmarks safety

One email a morning, machine-written

Related stories

CoMedBench: A Multi-Source Benchmark of Synthetic Medical Data Fidelity and Downstream Utility

Recovering Clinical Utility Under Differential Privacy: Empirical Validation of Adaptive Federated Aggregation on Heterogeneous Cardiovascular Datasets

Accelerating Reproducible Research in Synthetic EHR Generation

EHR-RobustGym: Benchmarking and Training Agents for Robust Clinical Reasoning

Synthetic Hospital: An Open, Verifiable, Physician-Validated Longitudinal EHR Benchmark

FoMoH: A clinically meaningful foundation model evaluation for structured electronic health records