Challenges of Auditing: Variability in Outputs of Large Language Models for Health
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
The Flow has not summarised this story yet — read it at arXiv Computation and Language.
arXiv:2606. 27302v1 Announce Type: cross Abstract: AI healthcare chatbots are increasingly used to support health information seeking and self-management, yet their performance and impact on users remains to be studied.
Users are asking language models mental health questions and are receiving answers. We audited what 41 language models recommend to users when the same clinical information reaches them from three foo...
arXiv:2609.18729v1 Announce Type: cross Abstract: Consumers increasingly use AI chatbots for advice on what to buy. With companies like OpenAI and Google monetising their AI through advertising, this...
SIM-UT is a multi‑turn, user‑simulation evaluation framework designed to uncover realistic failure modes in large‑language‑model (LLM) medical triage chatbots. The study applied SIM‑UT to three WhatsApp‑based health chatbots in India, South Africa, and Kenya, comparing a medically fine‑tuned model (MedGemma) with general‑purpose models (ChatGPT and Claude) across more than 5,000 multilingual simulated conversations. A prominent failure identified was de‑escalation risk, where chatbots incorrectly down‑rated serious symptoms after users minimized them, potentially leading to dangerous clinical outcomes.
arXiv:2607. 08625v1 Announce Type: new Abstract: Consumer-facing health chatbots powered by large language models (LLMs) are increasingly used for symptom assessment.
TherapyGym is a framework designed to evaluate and improve therapy chatbots on two key clinical dimensions: fidelity to evidence-based practice and safety. It uses the Cognitive Therapy Rating Scale (CTRS) to automatically score adherence to CBT techniques across multi‑turn sessions, and a multi‑label annotation scheme to assess therapy‑specific risks such as failure to address harm or abuse. The framework also introduces THERAPYJUDGEBENCH, a validation set of 116 dialogues with 1,270 expert ratings, and serves as a training harness where CTRS and safety rewards guide reinforcement learning with diverse patient simulations, leading to significant improvements in expert‑rated performance.