arXiv AI

Evaluating Health Misinformation in Low-Resource Languages: Integrating Small Language Models with a Culturally-Sensitive Responsible NLP Framework (Bangla as a Case Study)

arXiv:2607. 12336v1 Announce Type: cross Abstract: Artificial Intelligence (AI) technologies, while serving as a foundational enabler for modern social media and digital health services, exert a bivalent effect by simultaneously acting as a combatant against and a spread vector for misinformation.

arXiv Machine Learning
Jul 17

Addressing Benchmarking Gaps in Large Language Models for Health and Medicine with Dynamic Red-Teaming

arXiv:2508. 00923v3 Announce Type: replace Abstract: Large language models (LLMs) are increasingly used to answer health-related questions and support healthcare workflows, yet evidence for their safety still relies heavily on static benchmarks that can rapidly become obsolete or be optimized against.

By Jiazhen Pan (Cherise), Bailiang Jian (Cherise), Paul Hager (Cherise), Yundi Zhang (Cherise), Che Liu (Cherise), Friederike Jungmann (Cherise), Hongwei Bran Li (Cherise), Julian Canisius (Cherise), Chenyu You (Cherise), Junde Wu (Cherise), Jiayuan Zhu (Cherise), Fenglin Liu (Cherise), Yuyuan Liu (Cherise), Niklas Bubeck (Cherise), Moritz Knolle (Cherise), Chen (Cherise), Chen (Cherise), Christian Wachinger, Zhenyu Gong, Cheng Ouyang, Georgios Kaissis, Benedikt Wiestler, Daniel Rueckert
arXiv Computation and Language
Sep 18

HerHealthEval: Evaluating Multilingual and Register-Sensitive Understanding of Women's Health Communication

HerHealthEval is a controlled evaluation framework that tests multilingual understanding of women's-health communication across English, French, and Modern Standard Arabic. It provides six communicative forms—canonical, clinical, layperson, indirect or hedged, emotionally concerned, and deliberately under-specified—each conveying the same clinical concern except the under-specified form omits details to assess clarification needs. The study evaluates multilingual instruction models and QLoRA-adapted variants on tasks such as concern classification, risk calibration, clarification behavior, parse compliance, and cross-form consistency, revealing that high aggregate accuracy can mask safety-relevant failures and that language-invariant risk labels improve performance.

By Hassan Saeed Hassan Albattra, Mazen Mohammed Bahgat, Rahatara Ferdousi, Hana Essam Sayed Ahmed Amrya, Mariam Mousa
arXiv Computation and Language
Sep 2

Sources of Truth: A Multi-Platform, Multilingual Audit of Citations in AI Mental Health Information Queries

arXiv:2609.00319v1 Announce Type: cross Abstract: Online health information seeking is shifting from keyword search, where users consider a ranked list of links, to conversational systems that compos...

By Phuong Anh Nguyen, Jill Noorily, Matthew Flathers, Haruka Notsu, Laura Ospina-Pinillos, Tommy Nguyen, Samantha Clark, Aoife Keane, Grace Thompson, John Torous
arXiv AI
Sep 4

MIRA: A Bilingual Benchmark for Medical Information Response Audit

MIRA is a bilingual benchmark that evaluates whether large language models (LLMs) provide consistent medical information across different user phrasings, languages, and health literacy levels. It contains 4,320 prompts derived from 60 medically reviewed low‑risk health questions and reveals that models tend to omit key information and offer fewer concrete next steps when responding to low health‑literacy signals, a phenomenon termed Differential Information Dilution (DID). A knowledge‑guided mitigation prompt can reduce this dilution for most models, notably improving Claude and Qwen.

By Mengyu Xu, Qiaoxin Yang, Qianqian Wang, Xiwei Dai, Weiyi Wu, Chongyang Gao
arXiv Computation and Language
Sep 11

Complex-Text Robustness Evaluation and Failure Diagnosis for Low-Resource Multilingual Text-to-Speech

The paper introduces a framework to evaluate and diagnose the robustness of low‑resource multilingual text‑to‑speech systems when faced with complex text inputs such as numbers, dates, named entities, long sentences, code‑switched expressions, and punctuation structures. It assesses robustness across content consistency, language consistency, and generation stability, and proposes automatic metrics (character error rate, language ID accuracy, duration abnormal rate) along with a lightweight Text Risk Score (TRS) that predicts synthesis risk from interpretable text features. Experiments on Thai, Vietnamese, Swahili, and Indonesian TTS systems reveal distinct failure patterns and show that TRS correlates positively with content and duration errors, offering a low‑cost pre‑synthesis risk indicator.

By Tianlun Zuo, Ziyu Zhang, Tingzhi Mao, Zhonghua Fu, Lei Xie
arXiv Computation and Language
Aug 28

DocTalkBN: A Novel Dataset of Expert Telemedicine Conversations in Bengali

DocTalkBN is a large-scale multimodal dataset of authentic expert telemedicine conversations in Bengali, comprising 557.63 hours of paired audio and text, 1,515 multi-turn patient calls, and 10,274 host–doctor question–answer exchanges across 26 medical specialties. The dataset contains 1.7 million tokens and preserves the spontaneity and contextual richness of real medical interactions in a low-resource language. Three downstream tasks—medical triage classification, advice safety evaluation, and medical named entity recognition—are constructed to benchmark large language models and encoder-based baselines, demonstrating DocTalkBN’s practical usefulness for clinically grounded reasoning.

By Anik Saha, Fahmida Sultana Naznin, Sadatul Islam Sadi, Ananya Shahrin Promi, Wahid Al Azad Navid, Rifat Shahriyar
Hugging Face Trending Papers
Aug 3

An Evidence-Grounded Retrieval-Augmented Transformer Framework for Health Misinformation Verification

The rapid spread of false and misleading health information through digital platforms has become a major public health challenge, particularly during infectious disease outbreaks where delayed verification can influence public behaviour and hinder effective disease control. Although recent advances in automated health misinformation detection have shown encouraging results, most existing approaches rely heavily on global biomedical resources and often fail to capture the local context needed to verify claims in developing countries.