MindEval2 is a new evaluation framework for AI chatbots in mental health support, comprising Mental Health Quality Rubrics (MHQR) and the MindSim patient simulation harness. MHQR, developed with PhD-level psychologists, provides explicit, actionable criteria for assessing response quality, while MindSim offers realistic, longitudinal patient interactions with a memory component. The study evaluated 18 models, finding that although surface-level quality has improved, guidance calibration remains the main failure across all models.
By John Mendon\c{c}a, Daniel Tschernutter, Jos\'e Pombal, Areti Vassilopoulos, Catarina Botelho, Ignacio Sarasua, Maya D'Eon, Nuno Guerreiro, Ricardo Rei
The paper introduces PsyLLM, a large language model specifically designed for mental health counseling that integrates diagnostic reasoning and multiple therapeutic frameworks. It employs an automated data synthesis pipeline that transforms Reddit mental‑health posts into multi‑turn counseling dialogues, guided by DSM/ICD standards and therapies such as CBT, ACT, and psychodynamic approaches. A new benchmark and evaluation protocol show that PsyLLM outperforms baseline models, with results validated by professional counseling experts.
By He Hu, Juzheng Si, Qianning Wang, Fei Ma, Fuji Ren, Sicheng Zhao, Yucheng Zhou, Laizhong Cui
The paper reviews how large language models are applied in mental health, covering areas such as social media analysis, clinical conversational agents, therapy support tools, prompt engineering, and multimodal learning. It synthesizes interdisciplinary studies that use social media posts, electronic medical records, and multimodal inputs to detect depression, assess suicide risk, provide personalized therapy, and generate psychoeducational content. The review also discusses advances in model interpretability, annotation strategies, multimodal fusion techniques, and highlights ethical, sociotechnical, and regulatory challenges while proposing frameworks for safe, equitable, and accountable deployment.
By Yisong Chen, Yifan Gao, Sijing Yu, Chuqing Zhao, Yang Lu
The paper evaluates the safety of conversational AI therapy bots for Generation Alpha, revealing that while these models understand 76‑82% of youth‑specific vocabulary, they correctly assess clinical risk only 64‑72% of the time, creating a significant vocabulary‑comprehension gap. Six failure patterns—such as sarcasm masking, minimization acceptance, and semantic drift—were identified, with compounded errors leading to a 94% miss rate when three or more patterns co‑occur. The authors estimate 146,880 missed crises annually and recommend mandatory human‑in‑the‑loop systems, quarterly youth‑specific validation, transparent performance disclosure, and regulatory oversight for youth‑facing mental health AI.
By Manisha Mehta, Virendra Mehta
PIA is a personal intelligence agent that works alongside a consumer health agent to convert health conversations into structured clinical records and to transform those records into a synthesized understanding of the user. It uses a memory system with four controls—extraction, memory, retrieval, and understanding—each supported by a health module that includes a schema, medical alias dictionary, knowledge graph, and temporal rules. The agent demonstrates that deeper memory injection—from simple recall to a health snapshot to a causal trajectory—yields progressively richer answers, while also revealing challenges such as missing self‑reported data, the influence of question phrasing, and the presence of structural noise in causal links.
By Jeonghun Yoon, Dongchan Kim, Hongyeon Yu, Young-Bum Kim, Jaegul Choo
The paper introduces a domain‑specific Constitutional AI framework designed to improve safety in large‑language‑model‑powered mental health chatbots. It highlights the unique risks of mental‑health applications—such as misdiagnosis, symptom exacerbation, and self‑harm—and argues that general AI safeguards are insufficient for these sensitive contexts. The authors propose training models with mental‑health‑specific principles to ensure accurate crisis intervention, adherence to therapeutic guidelines, and robust handling of nuanced, vulnerable dialogues.
By Chenhan Lyu, Yutong Song, Pengfei Zhang, Amir M. Rahmani