SIM-UT is a multi‑turn, user‑simulation evaluation framework designed to uncover realistic failure modes in large‑language‑model (LLM) medical triage chatbots. The study applied SIM‑UT to three WhatsApp‑based health chatbots in India, South Africa, and Kenya, comparing a medically fine‑tuned model (MedGemma) with general‑purpose models (ChatGPT and Claude) across more than 5,000 multilingual simulated conversations. A prominent failure identified was de‑escalation risk, where chatbots incorrectly down‑rated serious symptoms after users minimized them, potentially leading to dangerous clinical outcomes.
arXiv:2607. 08625v1 Announce Type: new Abstract: Consumer-facing health chatbots powered by large language models (LLMs) are increasingly used for symptom assessment.
arXiv:2604.25415v2 Announce Type: replace-cross Abstract: People increasingly turn to general-purpose AI chatbots for advice about emotional and mental health problems, but the ability of these syste...
arXiv:2606. 30491v1 Announce Type: cross Abstract: Background.
The paper evaluates the safety of conversational AI therapy bots for Generation Alpha, revealing that while these models understand 76‑82% of youth‑specific vocabulary, they correctly assess clinical risk only 64‑72% of the time, creating a significant vocabulary‑comprehension gap. Six failure patterns—such as sarcasm masking, minimization acceptance, and semantic drift—were identified, with compounded errors leading to a 94% miss rate when three or more patterns co‑occur. The authors estimate 146,880 missed crises annually and recommend mandatory human‑in‑the‑loop systems, quarterly youth‑specific validation, transparent performance disclosure, and regulatory oversight for youth‑facing mental health AI.
MindEval2 is a new evaluation framework for AI chatbots in mental health support, comprising Mental Health Quality Rubrics (MHQR) and the MindSim patient simulation harness. MHQR, developed with PhD-level psychologists, provides explicit, actionable criteria for assessing response quality, while MindSim offers realistic, longitudinal patient interactions with a memory component. The study evaluated 18 models, finding that although surface-level quality has improved, guidance calibration remains the main failure across all models.
arXiv:2602. 05088v4 Announce Type: replace Abstract: Millions of people now use generative AI chatbots for psychological support.
arXiv:2607. 24817v1 Announce Type: cross Abstract: Digital mental health interventions (DMHIs) offer scalable support, but ensuring they accurately detect users' intent during volatile situations can be challenging.
arXiv:2608. 09939v1 Announce Type: cross Abstract: Production teams deploying LLM chat agents face a specific quality assurance gap: existing evaluation tools test individual responses or simulate social interactions, but none systematically verify whether real users can achieve their goals through multi-turn conversation.
The paper introduces a three-tier persona vector for user simulation in evaluating LLM agents, comprising 23 dimensions across demographics, behavioral traits, and emotional states, plus a query-complexity overlay. It demonstrates that these nuanced personas generate diverse, scenario-reactive conversations, leading to significant variations in agent goal achievement and compliance across different contexts. The model’s design allows for reproducible, auditable user behavior patterns without relying on learned covariance matrices.
arXiv:2605. 18937v2 Announce Type: replace Abstract: Patient-managed Personal Health Records (PHRs) promises to empower patients to better understand their health; but information in the record is complex, potentially hindering insights.
This scoping review examined 48 studies on generative AI chatbots designed to deliver motivational interviewing (MI). It found that most systems were text‑based and disembodied, with about half incorporating dynamic adaptation, and that safety reporting was inconsistent. While user perceptions were generally positive and many studies reported MI‑consistent interactions, evidence for sustained behavioral or functional change remains limited.
arXiv:2602.11391v5 Announce Type: replace Abstract: Objective: This study develops and validates a patient simulation framework that aligns with the National Institute of Standards and Technology AI...