arXiv Machine Learning July 28, 2026 By Zheng Hui, Yijiang River Dong, Ehsan Shareghi, Nigel Collier

TRIDENT: Benchmarking LLM Safety in Finance, Medicine, and Law

Read the original on arXiv Machine Learning →

arXiv:2507. 21134v2 Announce Type: replace-cross Abstract: As large language models (LLMs) are increasingly deployed in high-risk domains such as law, finance, and medicine, systematically evaluating their domain-specific safety and compliance becomes critical.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Jun 30

When Medical Safety Alignment Fails: A Benchmark for Evaluating LLMs on High-Risk Medical Queries

arXiv:2606. 28332v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used for medical and health-related questions, yet their safety in high-risk medical scenarios remains poorly understood.

By Yige Li, Jun Sun, Wei Zhao, Zhe Li, Yutao Wu, Hanxun Huang, Xiang Zheng, Xingjun Ma

llms fine-tuning benchmarks safety

arXiv AI

3d ago

ContextAdapt: Evaluating Contextual Adaptation and Value Alignment in LLMs

arXiv:2609.38260v1 Announce Type: cross Abstract: Values such as honesty, autonomy, and confidentiality are often regarded as general principles underpinning AI alignment. However, what it means to a...

By Olivia Macmillan-Scott, Mirco Musolesi

llms safety

arXiv Machine Learning

Sep 22

SafeTune: A Unified Faithful Library for Auditing and Repairing Safety Drift in Fine-Tuned LLMs

SafeTune is a source‑available library that consolidates four safety‑intervention paradigms—post‑hoc weight recovery, safety‑constrained fine‑tuning, gradient‑based unlearning, and inference‑time steering—into a single, configuration‑driven workflow. It offers shared interpretability, evaluation, and deployment tools, and its modular registry allows easy addition of new methods, benchmarks, judges, models, and fine‑tuning domains. The authors demonstrate SafeTune with controlled comparisons and case studies in finance and medical deployments, showing how it characterizes safety drift, evaluates interventions on refusal‑behavior and capability metrics, and supports calibrated or layered mitigation.

By Pratinav Seth, Saisab Sadhu, Anshul Kaushal, Vinay Kumar Sankarapu

llms fine-tuning benchmarks safety

arXiv AI

Jun 19

FFinRED: An Expert-Guided Benchmark Generation and Evaluation Framework for Financial LLM Red-Teaming

arXiv:2606. 19887v1 Announce Type: cross Abstract: Existing safety benchmarks target general adversarial scenarios but miss finance-specific risks.

By Chaeyun Kim, Daeyoung Park, Junghwan Kim, Jinyoung Jeong, Eunji Song, Yongtaek Lim, Minwoo Kim

llms benchmarks safety

arXiv AI

Sep 1

Risk-Adjusted Harm Scoring for Automated Red Teaming for LLMs in Financial Services

arXiv:2603.10807v2 Announce Type: replace-cross Abstract: Existing LLM safety evaluations rely on binary attack-success rates and domain-agnostic taxonomies, leaving regulated Banking, Financial Serv...

By Fabrizio Dimino, Bhaskarjit Sarmah, Stefano Pasquali

llms multimodal benchmarks safety

arXiv Computation and Language

Sep 22

Beyond Accuracy and Surface Fluency: Risk-Sensitive Evaluation of LLMs for Legal Clause Generation

arXiv:2609.22127v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used to draft contractual language, yet conventional accuracy or preference-based evaluations are poorly...

By Devansh Singh, Sundaraparipurnan Narayanan

llms

One email a morning, machine-written

Related stories

When Medical Safety Alignment Fails: A Benchmark for Evaluating LLMs on High-Risk Medical Queries

ContextAdapt: Evaluating Contextual Adaptation and Value Alignment in LLMs

SafeTune: A Unified Faithful Library for Auditing and Repairing Safety Drift in Fine-Tuned LLMs

FFinRED: An Expert-Guided Benchmark Generation and Evaluation Framework for Financial LLM Red-Teaming

Risk-Adjusted Harm Scoring for Automated Red Teaming for LLMs in Financial Services

Beyond Accuracy and Surface Fluency: Risk-Sensitive Evaluation of LLMs for Legal Clause Generation