arXiv AI
Context-Aware Classification and Grading of Sensitive Information in Online Conversational Health Data
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jun 6
Evaluating the Utility of Personal Health Records in Personalized Health AI
arXiv:2605. 18937v2 Announce Type: replace Abstract: Patient-managed Personal Health Records (PHRs) promises to empower patients to better understand their health; but information in the record is complex, potentially hindering insights.
arXiv Computation and Language
Aug 25
Clinically Grounded Privacy Evaluation of Medical LMs
The paper introduces a clinically grounded privacy evaluation framework for medical language models, assessing leakage across a spectrum of adversarial access levels—from publicly inferable demographics to leaked note fragments. Using this framework on an LM pretrained on 378,000 clinical notes, the authors find that routine encounter metadata leads to high verbatim memorization and significant recovery of sensitive diagnoses (e.g., AUROC 0.91 for abortion, 0.82 for HIV). They also note that exact-match memorization can overstate disclosure, with 36% of memorized tokens being templated documentation, underscoring the risks of training on longitudinal clinical data and offering a reusable evaluation tool.
arXiv AI
6d ago
AcuityBench: Evaluating Clinical Acuity Identification and Uncertainty Alignment
AcuityBench is a new benchmark that tests whether language models can correctly identify the urgency of medical care needed from user presentations. It unifies five public datasets—user conversations, online forum posts, clinical vignettes, and patient portal messages—under a shared four-level acuity framework, providing 914 cases for evaluation. The benchmark supports both explicit four-way classification and free-form conversational responses, revealing that models vary widely in accuracy and that conversational formats reduce over-triage but increase under-triage, especially for high-acuity cases.
arXiv AI
Jun 10
Expert-Level Crisis Detection in Mental Health Conversations
arXiv:2606. 10380v1 Announce Type: cross Abstract: Real-world crisis intervention is inherently conversational, yet existing research largely focuses on static texts.
arXiv Machine Learning
Aug 12
Automated Data Enrichment using Confidence-Aware Fine-Grained Debate among Open-Source LLMs for Mental Health and Online Safety
arXiv:2512. 06227v3 Announce Type: replace-cross Abstract: Real-world indicators play an important role in many Natural Language Processing (NLP) applications, such as life events for mental health analysis and risky behaviours for online safety, yet labelling such information is often costly and/or difficult due to its multi-label and dynamic nature.
arXiv AI
Jun 8
When Large Language Models Fail in Healthcare: Evaluating Sensitivity to Prompt Variations
arXiv:2606. 07237v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly used in healthcare for tasks such as clinical question answering, diagnosis support, and report summarization.