arXiv AI
TherapyGym: Evaluating and Aligning Clinical Fidelity and Safety in Therapy Chatbots
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Aug 24
When Vocabulary Comprehension Fails Clinical Reasoning: Evaluating Therapy Bots' Safety Risks for Generation Alpha
The paper evaluates the safety of conversational AI therapy bots for Generation Alpha, revealing that while these models understand 76‑82% of youth‑specific vocabulary, they correctly assess clinical risk only 64‑72% of the time, creating a significant vocabulary‑comprehension gap. Six failure patterns—such as sarcasm masking, minimization acceptance, and semantic drift—were identified, with compounded errors leading to a 94% miss rate when three or more patterns co‑occur. The authors estimate 146,880 missed crises annually and recommend mandatory human‑in‑the‑loop systems, quarterly youth‑specific validation, transparent performance disclosure, and regulatory oversight for youth‑facing mental health AI.
arXiv AI
Jul 1
Training Therapeutic Judges and Multi-Agent Systems for Human-Aligned Mental Health Support
arXiv:2606. 30887v1 Announce Type: cross Abstract: Large language models show promise for mental health support, yet therapeutic quality improves only when evaluation functions as an actionable control signal rather than a passive metric.
arXiv AI
Jul 9
AI Chatbot Suicide Risk Detection and Response: Human Validation Study of the Open-Source VERA-MH Safety Evaluation
arXiv:2602. 05088v4 Announce Type: replace Abstract: Millions of people now use generative AI chatbots for psychological support.
arXiv AI
Sep 15
K-Bench: a clinically calibrated benchmark for evaluating large language models in high-risk mental health conversations
arXiv:2609.15855v1 Announce Type: cross Abstract: % !TEX root = ../main.tex People increasingly use large language models (LLMs) for mental health support, yet their safety in evolving, high-risk con...
arXiv Computation and Language
Aug 31
Roleplaying with Structure: Synthetic Therapist-Client Conversation Generation from Questionnaires
arXiv:2510.25384v2 Announce Type: replace Abstract: Large Language Models (LLMs) are promising tools for synthetic data generation in mental health. However, privacy policies and restrictions forced...
arXiv Computation and Language
Oct 16
MindEval2: Clinically Grounded Patients and Rubrics for Evaluating Language Models in Mental Health Support
MindEval2 is a new evaluation framework for AI chatbots in mental health support, comprising Mental Health Quality Rubrics (MHQR) and the MindSim patient simulation harness. MHQR, developed with PhD-level psychologists, provides explicit, actionable criteria for assessing response quality, while MindSim offers realistic, longitudinal patient interactions with a memory component. The study evaluated 18 models, finding that although surface-level quality has improved, guidance calibration remains the main failure across all models.