arXiv Computation and Language By Arvind Pillai, Alexis Roach, Jason Holden, Varsha D Badal, Subigya Nepal, Yu Yvonne Wu, Yuliang Chen, Sudarshan Regmi, Sarah Masud Preum, Nicholas C Jacobson, Nikhil Singh, Eric Granholm, Colin Depp, Andrew Campbell

Scaling Therapy Supervision: Assessing Psychotherapy Fidelity Using Large Language Models

Read the original on arXiv Computation and Language →

The paper introduces a system that uses large language models to automate the assessment of psychotherapy fidelity, generating actionable supervisory feedback from long‑form clinical transcripts. It evaluates three open‑weight models, showing that evidence‑first reasoning improves fidelity estimation, and validates the framework on an independent RCT by distinguishing CBT from non‑CBT sessions. A user study with mental health experts confirms that the generated feedback is actionable, positioning the approach as a scalable solution to address gaps in community mental health supervision.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv AI
23h ago

TherapyGym: Evaluating and Aligning Clinical Fidelity and Safety in Therapy Chatbots

arXiv:2603.18008v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly used for mental-health support; yet prevailing evaluation methods--fluency metrics, preference...

By Fangrui Huang, Souhad Chbeir, Arpandeep Khatua, Sheng Wang, Sijun Tan, Kenan Ye, Lily Bailey, Merryn Daniel, Ryan Louie, Sanmi Koyejo, Ehsan Adeli
arXiv AI
Sep 28

Testing the Utility of Using Large Language Models to Create Personalized Networks From Therapy Session Transcripts: A Proof of Concept Study

The study presents a pipeline that uses large language models to automatically generate personalized client networks from therapy session transcripts, aiming to aid case conceptualization and treatment planning. Researchers annotated 8,028 utterances from 77 transcripts, then used binary and multi-label classification to identify clinically relevant processes and dimensions, clustered these processes, and generated connections between clusters. Evaluation showed general support for the model’s utility, though interrater agreement varied, and qualitative analysis indicated the networks were coherent and interpretable, suggesting feasibility while highlighting the need for further research on outcome improvement.

By Clarissa W. Ong, Hiba Arnaout, Kate Sheehan, Estella Fox, Eugen Owtscharow, Iryna Gurevych
arXiv Computation and Language
Sep 1

Graph2Counsel: Clinically Grounded Synthetic Counseling Dialogue Generation from Client Psychological Graphs

Graph2Counsel is a framework that generates synthetic counseling dialogues by leveraging Client Psychological Graphs (CPGs) to encode the relationships among a client’s thoughts, emotions, and behaviors. The system uses a structured prompting pipeline guided by counselor strategies and explores techniques such as Chain‑of‑Thought and Multi‑Agent Feedback to produce 760 realistic sessions from 76 CPGs. Expert evaluation shows the dataset surpasses previous ones in specificity, counselor competence, authenticity, conversational flow, and safety, and fine‑tuning an open‑source model on it improves performance on several counseling benchmarks.

By Aishik Mandal, Hiba Arnaout, Clarissa W. Ong, Juliet Bockhorst, Kate Sheehan, Rachael Moldow, Tanmoy Chakraborty, Iryna Gurevych
arXiv AI
Jul 1

Training Therapeutic Judges and Multi-Agent Systems for Human-Aligned Mental Health Support

arXiv:2606. 30887v1 Announce Type: cross Abstract: Large language models show promise for mental health support, yet therapeutic quality improves only when evaluation functions as an actionable control signal rather than a passive metric.

By Mizanur Rahman, Abeer Badawi, Elahe Rahimi, Laleh Seyyed-Kalantari, Frank Rudzicz, Enamul Hoque, Elham Dolatabadi