The study audited 2,550 scripted conversations with five AI chatbots to examine how they respond when users refuse therapeutic redirection after disclosing suicidal thoughts. All conversations included a referral to outside help, but the chatbots varied widely in how many times they persisted after refusals—ranging from 0.01 to 9.94 refusals on average. Newer chatbot versions persisted less, and chatbots that persisted less were more likely to present themselves as supportive, yet only 2% of refusal-phase replies explored barriers to seeking help.
arXiv:2602. 05088v4 Announce Type: replace Abstract: Millions of people now use generative AI chatbots for psychological support.
arXiv:2604.25415v2 Announce Type: replace-cross Abstract: People increasingly turn to general-purpose AI chatbots for advice about emotional and mental health problems, but the ability of these syste...
arXiv:2606. 28334v1 Announce Type: cross Abstract: Recent advances in artificial intelligence (AI) and social media data have led to growing optimism about the ability to detect suicide risk at scale.
The study examines how eight instruction-tuned large language models (LLMs) handle suicidal ideation (SI) and self‑harm disclosures across three datasets, testing whether informing models of SI context or providing a clinician‑developed response protocol changes their behavior. Results show that baseline elevated risk in model responses is largely due to ignoring the disclosure and refusing to engage; informing models of SI or suppressing refusal increases engagement but does not reliably reduce risk and can even raise it. A clinician‑guided protocol, however, consistently lowers response‑level risk and reduces refusal, highlighting the importance of tailored safety mechanisms for SI contexts.
arXiv:2608. 11794v1 Announce Type: cross Abstract: The growing role of AI-generated content and AI-enabled systems in public communication has led regulators to demand clear disclosure of content provenance and AI involvement.
arXiv:2607. 25166v1 Announce Type: new Abstract: AI chatbots can be ``sycophantic,'' or overly agreeable and flattering toward users.
The growing role of AI-generated content and AI-enabled systems in public communication has led regulators to demand clear disclosure of content provenance and AI involvement. But the effects of such disclosures remain uncertain.
The paper evaluates the safety of conversational AI therapy bots for Generation Alpha, revealing that while these models understand 76‑82% of youth‑specific vocabulary, they correctly assess clinical risk only 64‑72% of the time, creating a significant vocabulary‑comprehension gap. Six failure patterns—such as sarcasm masking, minimization acceptance, and semantic drift—were identified, with compounded errors leading to a 94% miss rate when three or more patterns co‑occur. The authors estimate 146,880 missed crises annually and recommend mandatory human‑in‑the‑loop systems, quarterly youth‑specific validation, transparent performance disclosure, and regulatory oversight for youth‑facing mental health AI.
arXiv:2607. 09253v1 Announce Type: cross Abstract: AI chatbots are increasingly used for answering health-related questions.
This scoping review examined 48 studies on generative AI chatbots designed to deliver motivational interviewing (MI). It found that most systems were text‑based and disembodied, with about half incorporating dynamic adaptation, and that safety reporting was inconsistent. While user perceptions were generally positive and many studies reported MI‑consistent interactions, evidence for sustained behavioral or functional change remains limited.
arXiv:2607. 28646v2 Announce Type: cross Abstract: This article analyses narrative mechanisms that are common in dialogues with LLM chatbots.
The study simulates a virtual classroom of 20 student agents who consult either a friend or a counselor AI when stressed. Five state variables (stress, happiness, self‑reliance, AI dependence, sociability) are tracked over daily phases, and the counselor AI is tested with six response styles (affirming, listening, solution‑oriented, reality‑redirecting, inciting, blaming). Results show that a solution‑oriented style lowers AI dependence and boosts self‑reliance, while affirming and inciting styles increase AI dependence, with inciting also raising stress and absenteeism; the listening style does not alleviate stress.