arXiv AI

When Medical Safety Alignment Fails: A Benchmark for Evaluating LLMs on High-Risk Medical Queries

arXiv:2606. 28332v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used for medical and health-related questions, yet their safety in high-risk medical scenarios remains poorly understood.

arXiv Computation and Language
Sep 3

HarmReduction: Benchmarking LLMs in Harm Reduction Information Provision to Support People Who Use Drugs

The paper introduces HarmReduction, a benchmark for evaluating large language models (LLMs) on their ability to provide accurate and safe harm reduction information to people who use drugs (PWUD). The benchmark, HR-Basic, contains 2,160 question‑answer‑evidence pairs covering safety boundary checks, quantitative value provision, and polysubstance risk inference. Experiments show that even state‑of‑the‑art LLMs struggle with accuracy and can pose severe safety risks, underscoring the need for a dedicated evaluation framework.

By Kaixuan Wang, Chenxin Diao, Jason T. Jacques, Zhongliang Guo, Shuai Zhao
arXiv Machine Learning
Jul 17

Addressing Benchmarking Gaps in Large Language Models for Health and Medicine with Dynamic Red-Teaming

arXiv:2508. 00923v3 Announce Type: replace Abstract: Large language models (LLMs) are increasingly used to answer health-related questions and support healthcare workflows, yet evidence for their safety still relies heavily on static benchmarks that can rapidly become obsolete or be optimized against.

By Jiazhen Pan (Cherise), Bailiang Jian (Cherise), Paul Hager (Cherise), Yundi Zhang (Cherise), Che Liu (Cherise), Friederike Jungmann (Cherise), Hongwei Bran Li (Cherise), Julian Canisius (Cherise), Chenyu You (Cherise), Junde Wu (Cherise), Jiayuan Zhu (Cherise), Fenglin Liu (Cherise), Yuyuan Liu (Cherise), Niklas Bubeck (Cherise), Moritz Knolle (Cherise), Chen (Cherise), Chen (Cherise), Christian Wachinger, Zhenyu Gong, Cheng Ouyang, Georgios Kaissis, Benedikt Wiestler, Daniel Rueckert
arXiv AI
Jul 22

MEDIC: Comprehensive Evaluation of Leading Indicators for LLM Safety and Utility in Clinical Applications

arXiv:2409. 07314v3 Announce Type: replace-cross Abstract: While Large Language Models (LLMs) achieve superhuman performance on standardized medical licensing exams, these static benchmarks have become saturated and increasingly disconnected from the functional requirements of clinical workflows.

By Praveenkumar Kanithi, Cl\'ement Christophe, Marco AF Pimentel, Tathagata Raha, Prateek Munjal, Nada Saadi, Hamza A Javed, Svetlana Maslenkova, Nasir Hayat, Ronnie Rajan, Shadab Khan
arXiv Machine Learning
Sep 22

SafeTune: A Unified Faithful Library for Auditing and Repairing Safety Drift in Fine-Tuned LLMs

SafeTune is a source‑available library that consolidates four safety‑intervention paradigms—post‑hoc weight recovery, safety‑constrained fine‑tuning, gradient‑based unlearning, and inference‑time steering—into a single, configuration‑driven workflow. It offers shared interpretability, evaluation, and deployment tools, and its modular registry allows easy addition of new methods, benchmarks, judges, models, and fine‑tuning domains. The authors demonstrate SafeTune with controlled comparisons and case studies in finance and medical deployments, showing how it characterizes safety drift, evaluates interventions on refusal‑behavior and capability metrics, and supports calibrated or layered mitigation.

By Pratinav Seth, Saisab Sadhu, Anshul Kaushal, Vinay Kumar Sankarapu
arXiv AI
Jun 17

First, do NOHARM: towards clinically safe large language models

arXiv:2512. 01241v3 Announce Type: replace-cross Abstract: Large language models (LLMs) are routinely used by physicians and patients for medical advice, yet their clinical safety profiles remain poorly characterized.

By David Wu, Fateme Nateghi Haredasht, Saloni Kumar Maharaj, Priyank Jain, Jessica Tran, Matthew Gwiazdon, Arjun Rustagi, Jenelle Jindal, Jacob M. Koshy, Vinay Kadiyala, Anup Agarwal, Bassman Tappuni, Brianna French, Sirus Jesudasen, Christopher V. Cosgriff, Rebanta Chakraborty, Jillian Caldwell, Susan Ziolkowski, David J. Iberri, Robert Diep, Rahul S. Dalal, Kira L. Newman, Kristin Galetta, J. Carl Pallais, Nancy Wei, Kathleen M. Buchheit, David I. Hong, Vartan Pahalyants, Ernest Y. Lee, Allen Shih, Tamara B. Kaplan, Vishnu Ravi, Sarita Khemani, Thomas A. Buckley, April S. Liang, Daniel Shirvani, Advait Patil, Nicholas Marshall, Kanav Chopra, Joel Koh, Adi Badhwar, Anastasia Perez, Austin J. Schoeffler, Mahbuba Tusty, Chase M. Walton, Liam G. McCoy, David J. H. Wu, Yingjie Weng, Sumant Ranji, Kevin Schulman, Nigam H. Shah, Jason Hom, Arnold Milstein, Arjun K. Manrai, Adam Rodman, Jonathan H. Chen, Ethan Goh