arXiv Computation and Language
Oct 5

Understanding Clinical Cognitive Dialogues Using Large Language Models

The paper introduces a de‑identified corpus of 33 in‑person cognitive assessment conversations, comprising 8,250 utterances annotated for three speaker roles and 56 dialogue acts. The authors benchmark large language models on fine‑grained dialogue‑act classification and next‑patient‑utterance generation, finding that instruction tuning and reasoning‑aware fine‑tuning improve performance but that models still struggle with closely related dialogue acts. The corpus and benchmark are presented as tools to measure interaction structure in cognitive assessments and to support future research on conversational markers, clinician education, and validated simulated patients.

By Vishalakshi Arumugam, Dan Schumacher, Veronica Rammouz, Erfan Nourbakhsh, Enrique Gonzalez Guerrero, Jeremy Davis, Anthony Rios
arXiv AI
Jun 17

AIPatient Arena: EHR-grounded evaluation of large language models in end-to-end clinical consultation workflows

arXiv:2606. 17474v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly considered for use in clinical consultation tasks, yet most medical evaluations remain static, single-turn, or narrowly outcome-based, limiting their ability to reflect the sequential, uncertain, and interactive nature of real-world care.

By Jiahui Niu, Huizi Yu, Wenkong Wang, Guangxin Dai, Jingxian He, Xiang Li, Zhiying Liang, Xinxin Lin, Kent CY So, Bryan YP Yan, Yun Kwok Wing, Yanqiu Xing, Xin Ma, Lizhou Fan
arXiv AI
Sep 12

Evaluating Scaffolding-Oriented Multi-Agent Large Language Model System for Clinical Interview Training

The study introduces a scaffolding-oriented multi-agent Large Language Model (LLM) AI Standardized Patient (AI-SP) platform designed to train medical students in patient interviews. In a randomized controlled trial with 100 students, the multi-agent system—comprising a patient agent, a Socratic tutor agent, and a turn-level evaluator—did not improve diagnostic accuracy but significantly enhanced overall OSCE scores, especially in communication, empathy, and history-taking. The authors also release a richly annotated dataset to support further research in AI-supported clinical reasoning training.

By Luming Yang, Haoxian Liu, Siqing Li, Rong Jia, Yue Xiao, Guanhua Chen, Li Lu