arXiv Computation and Language

Scaling Therapy Supervision: Assessing Psychotherapy Fidelity Using Large Language Models

The paper introduces a system that uses large language models to automate the assessment of psychotherapy fidelity, generating actionable supervisory feedback from long‑form clinical transcripts. It evaluates three open‑weight models, showing that evidence‑first reasoning improves fidelity estimation, and validates the framework on an independent RCT by distinguishing CBT from non‑CBT sessions. A user study with mental health experts confirms that the generated feedback is actionable, positioning the approach as a scalable solution to address gaps in community mental health supervision.

arXiv AI
23h ago

TherapyGym: Evaluating and Aligning Clinical Fidelity and Safety in Therapy Chatbots

arXiv:2603.18008v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly used for mental-health support; yet prevailing evaluation methods--fluency metrics, preference...

By Fangrui Huang, Souhad Chbeir, Arpandeep Khatua, Sheng Wang, Sijun Tan, Kenan Ye, Lily Bailey, Merryn Daniel, Ryan Louie, Sanmi Koyejo, Ehsan Adeli
arXiv AI
Sep 28

Testing the Utility of Using Large Language Models to Create Personalized Networks From Therapy Session Transcripts: A Proof of Concept Study

The study presents a pipeline that uses large language models to automatically generate personalized client networks from therapy session transcripts, aiming to aid case conceptualization and treatment planning. Researchers annotated 8,028 utterances from 77 transcripts, then used binary and multi-label classification to identify clinically relevant processes and dimensions, clustered these processes, and generated connections between clusters. Evaluation showed general support for the model’s utility, though interrater agreement varied, and qualitative analysis indicated the networks were coherent and interpretable, suggesting feasibility while highlighting the need for further research on outcome improvement.

By Clarissa W. Ong, Hiba Arnaout, Kate Sheehan, Estella Fox, Eugen Owtscharow, Iryna Gurevych
arXiv Computation and Language
Sep 1

Graph2Counsel: Clinically Grounded Synthetic Counseling Dialogue Generation from Client Psychological Graphs

Graph2Counsel is a framework that generates synthetic counseling dialogues by leveraging Client Psychological Graphs (CPGs) to encode the relationships among a client’s thoughts, emotions, and behaviors. The system uses a structured prompting pipeline guided by counselor strategies and explores techniques such as Chain‑of‑Thought and Multi‑Agent Feedback to produce 760 realistic sessions from 76 CPGs. Expert evaluation shows the dataset surpasses previous ones in specificity, counselor competence, authenticity, conversational flow, and safety, and fine‑tuning an open‑source model on it improves performance on several counseling benchmarks.

By Aishik Mandal, Hiba Arnaout, Clarissa W. Ong, Juliet Bockhorst, Kate Sheehan, Rachael Moldow, Tanmoy Chakraborty, Iryna Gurevych
arXiv AI
Jul 1

Training Therapeutic Judges and Multi-Agent Systems for Human-Aligned Mental Health Support

arXiv:2606. 30887v1 Announce Type: cross Abstract: Large language models show promise for mental health support, yet therapeutic quality improves only when evaluation functions as an actionable control signal rather than a passive metric.

By Mizanur Rahman, Abeer Badawi, Elahe Rahimi, Laleh Seyyed-Kalantari, Frank Rudzicz, Enamul Hoque, Elham Dolatabadi
arXiv AI
Sep 28

MACBT: A Multi-Agent Cognitive Behavioral Therapy Decision Support System with Longitudinal Memory

MACBT is a clinician‑facing AI decision‑support system that models the five‑stage CBT workflow with five collaborative agents and incorporates a CBT‑specific longitudinal memory module (CD Memory) to track cognitive distortions across sessions. The system generates pre‑session pathology reports and prioritises interventions, and was trained on a Chinese CBT dialogue corpus using a Qwen3‑14B backbone with supervised fine‑tuning and preference optimisation. Evaluation by GPT‑4 judges shows MACBT outperforms several existing chatbots in professionalism and clinical authenticity, and the memory‑augmented version improves session quality by 12.6% and achieves a longitudinal mean of 2.29 on continuity, progression, and personalization.

By De Jiang, Shuo Zhang, Weiwei Liao, Jianying Zhang, Chuanhui Yu, Hongen Liao, Kehong Yuan
arXiv Computation and Language
Aug 31

Roleplaying with Structure: Synthetic Therapist-Client Conversation Generation from Questionnaires

arXiv:2510.25384v2 Announce Type: replace Abstract: Large Language Models (LLMs) are promising tools for synthetic data generation in mental health. However, privacy policies and restrictions forced...

By Doan Nam Long Vu, Rui Tan, Lena Moench, Svenja Jule Francke, Daniel Woiwod, Florian Thomas-Odenthal, Sanna Stroth, Tilo Kircher, Christiane Hermann, Udo Dannlowski, Hamidreza Jamalabadi, Simone Balloccu, Shaoxiong Ji
arXiv AI
Jul 10

MentalHospital: A Virtual Environment for Evaluating Psychiatric Clinical Encounters

arXiv:2607. 08257v1 Announce Type: new Abstract: Large language models (LLMs) have shown strong performance on isolated psychiatric tasks, including dialogue, diagnosis, and treatment planning, yet existing benchmarks rarely simulate complete psychiatric clinical encounters.

By Yuming Yang, Xiao Sun, Yuanwei Zou, Zhengxiao Wu, Yun Chen, Jiang Zhong, Haoyang Zeng, Jingwang Huang, Kaiwen Wei