arXiv Computation and Language
Sep 17

"If I Had to Buy Just ONE: Galaxy S26 Ultra": Auditing AI-Generated Product Recommendations

arXiv:2609.18729v1 Announce Type: cross Abstract: Consumers increasingly use AI chatbots for advice on what to buy. With companies like OpenAI and Google monetising their AI through advertising, this...

By Lucas G. Uberti-Bona Marin, Thales Bertaglia, Giovanni Astante, Bram Rijsbosch, Gijs van Dijck, Anik\'o Hann\'ak, Gerasimos Spanakis, Konrad Kollnig
arXiv Computation and Language
Oct 23

SIM-UT: A User Simulation based Evaluation Framework for Medical Triage Chatbots

SIM-UT is a multi‑turn, user‑simulation evaluation framework designed to uncover realistic failure modes in large‑language‑model (LLM) medical triage chatbots. The study applied SIM‑UT to three WhatsApp‑based health chatbots in India, South Africa, and Kenya, comparing a medically fine‑tuned model (MedGemma) with general‑purpose models (ChatGPT and Claude) across more than 5,000 multilingual simulated conversations. A prominent failure identified was de‑escalation risk, where chatbots incorrectly down‑rated serious symptoms after users minimized them, potentially leading to dangerous clinical outcomes.

By Deepali Mittal, Dheeraj Mundhra, Yulia Sidorova, Tersia de Kock, Oli Munns, Sofie S Meyer
arXiv AI
Nov 5

TherapyGym: Evaluating and Aligning Clinical Fidelity and Safety in Therapy Chatbots

TherapyGym is a framework designed to evaluate and improve therapy chatbots on two key clinical dimensions: fidelity to evidence-based practice and safety. It uses the Cognitive Therapy Rating Scale (CTRS) to automatically score adherence to CBT techniques across multi‑turn sessions, and a multi‑label annotation scheme to assess therapy‑specific risks such as failure to address harm or abuse. The framework also introduces THERAPYJUDGEBENCH, a validation set of 116 dialogues with 1,270 expert ratings, and serves as a training harness where CTRS and safety rewards guide reinforcement learning with diverse patient simulations, leading to significant improvements in expert‑rated performance.

By Fangrui Huang, Souhad Chbeir, Arpandeep Khatua, Sheng Wang, Sijun Tan, Kenan Ye, Lily Bailey, Merryn Daniel, Ryan Louie, Sanmi Koyejo, Ehsan Adeli