HerHealthEval is a controlled evaluation framework that tests multilingual understanding of women's-health communication across English, French, and Modern Standard Arabic. It provides six communicative forms—canonical, clinical, layperson, indirect or hedged, emotionally concerned, and deliberately under-specified—each conveying the same clinical concern except the under-specified form omits details to assess clarification needs. The study evaluates multilingual instruction models and QLoRA-adapted variants on tasks such as concern classification, risk calibration, clarification behavior, parse compliance, and cross-form consistency, revealing that high aggregate accuracy can mask safety-relevant failures and that language-invariant risk labels improve performance.
By Hassan Saeed Hassan Albattra, Mazen Mohammed Bahgat, Rahatara Ferdousi, Hana Essam Sayed Ahmed Amrya, Mariam Mousa
The article investigates whether commercial machine translation (MT) tools—Google Translate, Microsoft Translator, GPT, Gemini, and Claude—are ready to support communication during crises or pandemics. Using the TICO-19 dataset, which contains pandemic-related sentences in high-priority low-resource languages, the study evaluates the usability of translations produced by these systems. The goal is to determine how effectively these MT tools can bridge language gaps between governments, aid providers, medical professionals, and affected communities.
By Vipasha Bansal, William D. Lewis, Nicole Mihaylova, Elizabeth Brown, Benjamin Pong
arXiv:2503.22735v2 Announce Type: replace-cross
Abstract: This article reports on the third iteration of a survey of computerized tools and technologies taught as part of postgraduate translation tra...
By Andrew Rothwell, Joss Moorkens, Tomas Svoboda
arXiv:2608. 19981v1 Announce Type: new Abstract: We present HealMed, an expert-reviewed benchmark for multilingual evaluation of large language models in medicine.
By Yingjian Chen (Drew), Fan Gao (Drew), Sherry T. Tong (Drew), Haoyu Zhang (Drew), Aosong Feng (Drew), Kevin W. Jin (Drew), Xing Wu (Drew), Jinghui Lu (Drew), Abdul Samad (Drew), Akbar Faruqi (Drew), Cesar Caraballo (Drew), Cibele Brand\~ao (Drew), Dhruva (Drew), Gupta, Eunji Jeon, Gabriel Madera-Santiago, Geon Lee, Hugo Toshio Itikawa, Insook Cho, Isabelli Martins, Isarar Siddique, Israr Ahmed, Jihyo Kwak, Kanyakorn Veerakanjana, Luis Guilherme Cardoso, Minjin Kim, Piyalitt Ittichaiwong, Renee Dua, Santiago Gudi\~no-Rosales, Xiujie Chen, Zeo Lapalus, Zixin Xu, Michihiro Yasunaga, Rex Ying, Heuiseok Lim, Jaewoo Kang, Chanjun Park, Hang Jiang, Ethan Goh, Hyunjae Kim, Edison Marrese-Taylor, Yusuke Iwasawa, Yutaka Matsuo, Qingyu Chen, Irene Li
arXiv:2610.32842v1 Announce Type: new
Abstract: It is essential that we communicate with the entirety of an affected population during a crisis, including those who do not speak a region's dominant l...
By Jen Wilson, Vipasha Bansal, William D. Lewis
arXiv:2609.17544v1 Announce Type: new
Abstract: Large language models (LLMs) are increasingly being explored for clinical applications, yet their assessment for real-world traditional Chinese medicin...
By Jiacheng Xie, Xiaoting Tang, Yang Yu, Jinpu Li, Shouli Li, Congcong Jing, Yantao Yang, Zhiyong Zhao, Ziyang Zhang, Qilin Song, Guanghui An, Dong Xu