← Back to all news
arXiv Computation and Language By Chenhan Lyu, Yutong Song, Pengfei Zhang, Amir M. Rahmani

Domain-Specific Constitutional AI: Enhancing Safety in LLM-Powered Mental Health Chatbots

Read the original on arXiv Computation and Language →

The paper introduces a domain‑specific Constitutional AI framework designed to improve safety in large‑language‑model‑powered mental health chatbots. It highlights the unique risks of mental‑health applications—such as misdiagnosis, symptom exacerbation, and self‑harm—and argues that general AI safeguards are insufficient for these sensitive contexts. The authors propose training models with mental‑health‑specific principles to ensure accurate crisis intervention, adherence to therapeutic guidelines, and robust handling of nuanced, vulnerable dialogues.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv AI
Aug 24

When Vocabulary Comprehension Fails Clinical Reasoning: Evaluating Therapy Bots' Safety Risks for Generation Alpha

The paper evaluates the safety of conversational AI therapy bots for Generation Alpha, revealing that while these models understand 76‑82% of youth‑specific vocabulary, they correctly assess clinical risk only 64‑72% of the time, creating a significant vocabulary‑comprehension gap. Six failure patterns—such as sarcasm masking, minimization acceptance, and semantic drift—were identified, with compounded errors leading to a 94% miss rate when three or more patterns co‑occur. The authors estimate 146,880 missed crises annually and recommend mandatory human‑in‑the‑loop systems, quarterly youth‑specific validation, transparent performance disclosure, and regulatory oversight for youth‑facing mental health AI.

By Manisha Mehta, Virendra Mehta
llmsbenchmarkssafety
More like this →
arXiv Computation and Language
Sep 23

From Pattern Recognizers to Personalized Companions: A Survey of Large Language Models in Mental Health

The article surveys how large language models (LLMs) are being applied to mental health, outlining a three‑phase evolution: Phase I uses LLMs as passive information tools and pattern recognizers for assessment; Phase II employs them as empathetic conversationalists for stateless, in‑the‑moment interactions; Phase III aims to create longitudinal, personalized companions that act as stateful cognitive agents. It systematically reviews core technologies, agent architectures (Profile, Memory, Reasoning, Planning), and the datasets and benchmarks that support this progression, offering a coherent narrative and roadmap for future research. The survey also provides a curated resource list at https://github.com/Emo-gml/Awesome-Mental-Health-LLMs.

By He Hu, Yucheng Zhou, Qianning Wang, Yingjian Zou, Chiyuan Ma, Juzheng Si, Jianzhuang Liu, Zitong Yu, Laizhong Cui, Fei Ma, Qi Tian
llmsagentsbenchmarkssafety
More like this →