VeriSim: A Configurable Framework for Stress-Testing Medical AI Under Patient Communication Noise

VeriSim is a configurable patient simulation framework that injects controllable noise across six clinically grounded communication dimensions while preserving patient medical records. It uses a verifier that extracts atomic claims from utterances and checks them against a UMLS‑grounded vector index built with BioLORD embeddings, relying on structured clinical metadata rather than surface‑text similarity. Stress‑testing seven open‑weight LLMs shows realistic noise reduces diagnostic accuracy by 15‑25 percentage points and lengthens conversations by 34‑55%, with smaller models degrading more than larger ones; expert ratings confirm VeriSim’s high truth, realism, clinical utility, and noise fidelity.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• llms
• rag