arXiv AI
Jun 4
AICompanionBench: Benchmarking LLMs-as-Judges for AI Companion Safety
arXiv:2606. 04867v1 Announce Type: new Abstract: As AI companion platforms such as Replika and Character.
AI Chatbot Suicide Risk Detection and Response: Human Validation Study of the Open-Source VERA-MH Safety Evaluation
arXiv:2602. 05088v4 Announce Type: replace Abstract: Millions of people now use generative AI chatbots for psychological support.
Related stories
arXiv AI
Jul 28
Risk Governance for Generative AI Mental Health Support: A Multi-Turn Safety Architecture
arXiv:2607. 22692v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used for emotional support despite lacking mechanisms to safely govern evolving mental health risk.
arXiv AI
Jul 15
First, do NOHARM: a medical safety benchmark and randomized study of physician and AI teaming on clinical consultations
arXiv:2512. 01241v4 Announce Type: replace-cross Abstract: Large language models (LLMs) and medical AI tools are routinely used by physicians and patients for medical advice, yet their clinical safety profiles remain poorly characterized.
arXiv AI
Jul 29
PatientAgentBench: A Benchmark Framework for Evaluating Patient-Facing Health AI Agents
arXiv:2607. 25485v1 Announce Type: new Abstract: Health AI is evolving from answering questions to agentic systems that converse with patients, reason about health records, and act on their behalf.
arXiv AI
Jun 30
Ground Truths in Suicide Research: The Current State of AI-Based Suicide Detection in Social Media
arXiv:2606. 28334v1 Announce Type: cross Abstract: Recent advances in artificial intelligence (AI) and social media data have led to growing optimism about the ability to detect suicide risk at scale.
arXiv AI
Jul 29
Retrieval-Augmented Generation in LLMs for Mental Health: Quantifying the Incremental Contribution of Retrieval Within a Layered Safety Architecture
arXiv:2607. 24817v1 Announce Type: cross Abstract: Digital mental health interventions (DMHIs) offer scalable support, but ensuring they accurately detect users' intent during volatile situations can be challenging.
arXiv AI
Jun 30
EMPATH: A Multilingual Auditor-Judge Benchmark for Safety Evaluation of Emotional-Support Chatbots
arXiv:2606. 30256v1 Announce Type: new Abstract: Safety benchmarks often buy scalability by fixing the prompt, the language, and the turn structure.
arXiv AI
Jul 10
The complexities of patient-centred conversational artificial intelligence
arXiv:2607. 08625v1 Announce Type: new Abstract: Consumer-facing health chatbots powered by large language models (LLMs) are increasingly used for symptom assessment.
arXiv AI
Jul 29
Psychological Influences of Conversational AI: Research and Design Directions for Reducing Harm and Promoting Well-Being
arXiv:2607. 25057v1 Announce Type: new Abstract: As conversational AI systems become increasingly integrated into daily life, their potential effects on user well-being require ongoing attention.
arXiv AI
Jun 17
First, do NOHARM: towards clinically safe large language models
arXiv:2512. 01241v3 Announce Type: replace-cross Abstract: Large language models (LLMs) are routinely used by physicians and patients for medical advice, yet their clinical safety profiles remain poorly characterized.
arXiv AI
Aug 11
Beyond "I Can't Help With That": How Child Safety Experts Evaluate AI Chatbot Safety
arXiv:2608. 07902v1 Announce Type: cross Abstract: Youth increasingly turn to AI chatbots for social and emotional support, raising concerns about how these systems respond, especially in high-stakes situations.
arXiv AI
Jul 1
Training Therapeutic Judges and Multi-Agent Systems for Human-Aligned Mental Health Support
arXiv:2606. 30887v1 Announce Type: cross Abstract: Large language models show promise for mental health support, yet therapeutic quality improves only when evaluation functions as an actionable control signal rather than a passive metric.