MBBQ (Multilingual Bias Benchmark for Question-answering) is a curated extension of the English BBQ dataset that includes Dutch, Spanish, and Turkish, designed to measure stereotypes across these languages while controlling for cultural differences and task accuracy. The dataset also provides a parallel control set to assess question-answering performance independently of bias. Experiments with several open-source and proprietary LLMs show that non‑English languages can exhibit greater bias than English, and cross‑lingual bias differences persist except in the most accurate models.
By Vera Neplenbroek, Arianna Bisazza, Raquel Fern\'andez
The paper introduces TriviaRoomQA, a multilingual benchmark of 3,300 parallel multiple-choice questions across 288 topics, plus an additional 5,340 French-only questions. It evaluates 30 open-weight large language models (7–70 B parameters) from various regions, finding strong performance on knowledge-intensive subjects like history and geography but weaker results on everyday popular‑culture topics such as celebrities, music, movies, and news. The study also shows that model performance varies across languages for the same questions, indicating that factual knowledge access is not always language‑independent.
By Anna Mosolova, Djam\'e Seddah
Quiz rooms, trivia nights, and quiz shows challenge human knowledge across a wide range of topics, from canonical facts to everyday culture. In this paper, we examine whether large language models (LLMs) can perform competitively in such settings, using quiz-style questions to test them on both common and niche topics.
arXiv:2609.38256v1 Announce Type: new
Abstract: Large language models (LLMs) are increasingly used to answer questions about politically contentious issues, yet evaluations typically treat a model's...
By Olivia Macmillan-Scott, Michael Jacobs, Nils Metternich, Mirco Musolesi
The study examines how language influences AI chatbot responses to questions about the war in Ukraine, revealing that the same AI systems (GPT, Claude, Gemini) produce varying political stances across 112 languages. By evaluating 20 statements in 112 languages, the researchers found that Russia‑leaning versus Ukraine‑leaning answers differ by language, mirroring global political attitudes such as public support for Russia, UN voting patterns, and aid levels. This pattern persists across all three models and even when specific statement pairs are removed, suggesting that information warfare could embed geopolitical biases into AI training data.
By Maxim Chupilkin
arXiv:2603. 23841v2 Announce Type: replace-cross Abstract: While Large Language Models (LLMs) are increasingly used as primary sources of information, their potential for political bias may impact their objectivity.
By Rohan Khetan, Ashna Khetan
arXiv:2609.27811v1 Announce Type: cross
Abstract: Online platforms have become arenas for the public contestation of climate change, shaping how scientific knowledge, denial, and uncertainty are expr...
By Daniel Morais, Diego H. M. Magalhaes, Gabriel H. Silva, Andrea Failla, Valeria de C. Santos, Helen C. S. C. Lima, Carlos H. G. Ferreira
The paper investigates position bias in large language models (LLMs) across a wide range of model sizes—from billions to trillions of parameters—and across five languages. It finds that while small-scale models and non‑English languages still exhibit noticeable position bias, larger models and the use of Chain‑of‑Thought prompting substantially reduce this bias. Additionally, the study shows that emerging thinking models with advanced reasoning capabilities are almost immune to position bias, indicating a promising but resource‑intensive path to mitigating this vulnerability.
By Yining Li, Huayang Li, Lemao Liu, Taro Watanabe
arXiv:2605.12515v3 Announce Type: replace
Abstract: Multilingual large language models (MLLMs) often answer culturally ambiguous questions inconsistently across languages, even for a fixed user perso...
By Lucas Resck, Isabelle Augenstein, Anna Korhonen
arXiv:2607. 19243v1 Announce Type: cross Abstract: Although Large Language Models (LLMs) demonstrate remarkable multilingual fluency, their internal knowledge representations remain disproportionately biased toward high-resource languages.
By Alexander Manev
arXiv:2607. 22699v1 Announce Type: new Abstract: As Large Language Models (LLMs) grow more capable across diverse tasks, their (in)ability to generalize remains difficult to quantify and poorly understood beyond limited domains.
By Supantho Rakshit, Adele Goldberg, Henry Conklin
arXiv:2609.37498v1 Announce Type: new
Abstract: Wikipedia aims to provide a unified, neutral record of history, yet its independent language editions often function as distinct epistemic communities,...
By Claudiu Creanga, Liviu P. Dinu, Anca Dinu