arXiv AI
Right Words, Wrong Moment: A Clinician-Grounded Analysis of Distress in 19,930 Conversations between Young People and ChatGPT
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
OpenAI Blog
Oct 27, 2025
Strengthening ChatGPT’s responses in sensitive conversations
OpenAI collaborated with 170+ mental health experts to improve ChatGPT’s ability to recognize distress, respond empathetically, and guide users toward real-world support—reducing unsafe responses by up to 80%. Learn how we’re making ChatGPT safer and more supportive in sensitive moments.
arXiv AI
4d ago
Single-turn emergency psychiatric triage across 15 frontier AI chatbots
arXiv:2604.25415v2 Announce Type: replace-cross Abstract: People increasingly turn to general-purpose AI chatbots for advice about emotional and mental health problems, but the ability of these syste...
arXiv AI
Jul 28
Risk Governance for Generative AI Mental Health Support: A Multi-Turn Safety Architecture
arXiv:2607. 22692v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used for emotional support despite lacking mechanisms to safely govern evolving mental health risk.
arXiv AI
Sep 16
From Momentary Emotion Inference to Sustained Emotion Support: Evaluating a Companion Agent in a Longitudinal Study
The study evaluates PAIR, a theory-based emotion‑regulation companion, over 14 days with 19 participants, analyzing 1,093 sessions. Emotion estimates from the agent aligned better with participants’ self‑reported valence and dominance than arousal, and guided conversations led to higher valence and state‑dependent arousal changes. Participants reported feeling understood, and the perceived helpfulness of guided conversations increased over time, highlighting the role of memory updates and cross‑session personalization in sustained emotional support.
arXiv AI
Aug 11
Beyond "I Can't Help With That": How Child Safety Experts Evaluate AI Chatbot Safety
arXiv:2608. 07902v1 Announce Type: cross Abstract: Youth increasingly turn to AI chatbots for social and emotional support, raising concerns about how these systems respond, especially in high-stakes situations.
arXiv Computation and Language
Sep 21
Generative Artificial Intelligence Chatbots for Motivational Interviewing: A Scoping Review From System Design to Intervention Outcomes
This scoping review examined 48 studies on generative AI chatbots designed to deliver motivational interviewing (MI). It found that most systems were text‑based and disembodied, with about half incorporating dynamic adaptation, and that safety reporting was inconsistent. While user perceptions were generally positive and many studies reported MI‑consistent interactions, evidence for sustained behavioral or functional change remains limited.