The paper introduces WinoQueer-NL, a Dutch adaptation of the English WinoQueer benchmark, designed to assess anti‑queer bias in Dutch language models. After validating the dataset with 43 queer Dutch participants, the authors expanded it to 42,906 sentences and evaluated several Dutch and multilingual models, finding that while overall bias scores appeared neutral, specific identities—particularly transgender and non‑binary—were disproportionately favored in stereotypical sentences. The study underscores the need for culturally grounded datasets to identify and mitigate biases that affect marginalized groups in Dutch NLP systems.
By Jiska Beuk, Gerasimos Spanakis
The paper introduces WinoQueer‑NL, a Dutch adaptation of the English WinoQueer benchmark, designed to assess anti‑queer bias in Dutch language models. After validating the dataset with 43 queer Dutch participants, the authors released 42,906 sentences and evaluated several Dutch‑specific and multilingual models, finding that while overall bias scores were neutral, certain models disproportionately favored stereotypical statements for transgender and non‑binary identities. The study underscores the need for culturally grounded datasets to identify and mitigate biases that affect marginalized groups in Dutch NLP systems.
The paper introduces GPTBIAS, a framework that uses powerful large language models like GPT‑4 to evaluate bias in other LLMs. It employs specially crafted prompts called Bias Attack Instructions to probe for bias and outputs a bias score along with detailed information such as bias types, affected demographics, keywords, reasons, and improvement suggestions. Extensive experiments demonstrate the framework’s effectiveness and usability.
By Jiaxu Zhao, Meng Fang, Shirui Pan, Wenpeng Yin, Mykola Pechenizkiy
MBBQ (Multilingual Bias Benchmark for Question-answering) is a curated extension of the English BBQ dataset that includes Dutch, Spanish, and Turkish, designed to measure stereotypes across these languages while controlling for cultural differences and task accuracy. The dataset also provides a parallel control set to assess question-answering performance independently of bias. Experiments with several open-source and proprietary LLMs show that non‑English languages can exhibit greater bias than English, and cross‑lingual bias differences persist except in the most accurate models.
By Vera Neplenbroek, Arianna Bisazza, Raquel Fern\'andez
The paper introduces a scope‑conditioned generation framework that incorporates structured stereotype characteristics into prompts for large language models, aiming to improve the quality of counterspeech against online hate speech. The authors validate the method on a new, human‑curated dataset in English, Italian, and Spanish, showing significant gains over generic baselines in factuality, specificity, cogency, and effectiveness for both explicit and implicit stereotypes.
By Greta Damo, Elias Urios Alacreu, Elena Cabrio, Paolo Rosso, Serena Villata
arXiv:2610.27285v1 Announce Type: cross
Abstract: Effective detection of hate content is essential for maintaining healthy online communities, yet performance can vary substantially across languages....
By Zahra Bokaei, Walid Magdy, Bonnie Webber
arXiv:2604.20677v3 Announce Type: replace
Abstract: Large Language Models (LLMs) are increasingly deployed in socially sensitive settings, raising concerns about fairness and bias, particularly when...
By Chaima Boufaied, Ronnie De Souza Santos, Ann Barcomb
arXiv:2512. 15792v3 Announce Type: replace-cross Abstract: Large language models (LLMs) have rapidly become indispensable tools for acquiring information and supporting human decision-making.
By Xulang Zhang, Rui Mao, Erik Cambria
A common approach to measuring bias in Large Language Models is to compare the log-likelihoods of two contrastive stereotype sentences. We argue that such single-pair comparisons are often unreliable:...
The paper investigates gender bias in machine translation evaluation metrics using an occupation-balanced subset of GAMBIT+ across seven English‑source language pairs, including a new German extension. It finds that masculine translations tend to receive higher scores and that biases align with stereotypical gender representations, though the strength varies by evaluator and language. The study highlights that assessing bias requires multiple dimensions beyond a single aggregate measure.
By Orfeas Menis Mastromichalakis, Giorgos Filandrianos, Wafaa Mohammed, Giuseppe Attanasio, Chrysoula Zerva
arXiv:2512. 15792v4 Announce Type: replace-cross Abstract: Large language models (LLMs) have rapidly become indispensable tools for acquiring information and supporting human decision-making.
By Xulang Zhang, Rui Mao, Erik Cambria
Counterspeech (CS) - direct responses that counter online Hate Speech (HS) using reasoning and alternative viewpoints - has emerged as an alternative to content removal. Current automatic CS generatio...