MIRA is a bilingual benchmark that evaluates whether large language models (LLMs) provide consistent medical information across different user phrasings, languages, and health literacy levels. It contains 4,320 prompts derived from 60 medically reviewed low‑risk health questions and reveals that models tend to omit key information and offer fewer concrete next steps when responding to low health‑literacy signals, a phenomenon termed Differential Information Dilution (DID). A knowledge‑guided mitigation prompt can reduce this dilution for most models, notably improving Claude and Qwen.
By Mengyu Xu, Qiaoxin Yang, Qianqian Wang, Xiwei Dai, Weiyi Wu, Chongyang Gao
CrisiLangs is a multilingual extension of the CrisiText dataset, adding Italian, Bulgarian, German, and Greek translations of warning messages for 1,000 crisis events, resulting in 5,000 event‑message pairs across five languages. The resource was used to fine‑tune a multilingual model, which outperformed a zero‑shot baseline on ROUGE‑L, BLEU, and BERTScore in every language and configuration. Human preference studies showed that low decoding temperatures were favored across all languages, while the impact of emergency guidelines varied by language.
By Giacomo Gonella, Stefano Menini, Marco Guerini
arXiv:2511.00421v2 Announce Type: replace-cross
Abstract: Large language models (LLMs) show promise in medical applications, but their ability to detect and correct errors in clinical texts remains u...
By Naoto Iwase, Hiroki Okuyama, Junichiro Iwasawa
arXiv:2605.05662v2 Announce Type: replace-cross
Abstract: Current LLM safety benchmarks are predominantly English-centric and often rely on translation, failing to capture country-specific harms. Mor...
By Dasol Choi, Eugenia Kim, Jaewon Noh, Sang Seo, Eunmi Kim, Myunggyo Oh, Yunjin Park, Brigitta Jesica Kartono, Josef Pichlmeier, Helena Berndt, Sai Krishna Mendu, Glenn Johannes Tungka, \"Ozlem G\"ok\c{c}e, Suresh Gehlot, Katherine Pratt, Amanda Minnich, Haon Park
The study evaluates whether a single confidence threshold in typed decision interfaces treats all languages equally. Using 1,000 items in 51 languages across three vendors, it finds that English consistently achieves higher coverage—clearing the 0.9 threshold 8.6–11.7 percentage points more often than other languages—despite similar accuracy. Even when thresholds are re‑weighted or calibrated per language, the gap persists, and translating inputs into English reduces but does not eliminate it. The authors recommend reporting per‑language coverage at the operating threshold.
By Junwei Yao, Yongxi Zhou, Zike Yang, Zhijun Wang, Jianan Liu, Yanxian Chen, Wenbo Ye
arXiv:2606. 07853v1 Announce Type: cross Abstract: Large Language Models are transforming the support for clinical decision and their application in real scenarios.
By Giordano de Pinho Souza, Glaucia Melo, Josefino Cabral Melo Lima, Daniel Schneider