The study investigates cultural biases in large language models (LLMs) by testing their ability to perform author profiling—inferring singers’ gender and ethnicity—from song lyrics in a zero‑shot setting. Evaluating over 10,000 lyrics across several open‑source models, the authors find that most LLMs default toward North American ethnicity, while DeepSeek‑1.5B leans toward Asian ethnicity, and that Ministral‑8B exhibits the strongest ethnicity bias whereas Gemma‑12B is the most balanced. The paper introduces two fairness metrics, Modality Accuracy Divergence (MAD) and Recall Divergence (RD), to quantify these disparities and provides code and results publicly on GitHub and HuggingFace.
By Valentin Lafargue, Ariel Guerra-Adames, Emmanuelle Claeys, Elouan Vuichard, Jean-Michel Loubes
A common approach to measuring bias in Large Language Models is to compare the log-likelihoods of two contrastive stereotype sentences. We argue that such single-pair comparisons are often unreliable:...
arXiv:2610.08747v1 Announce Type: new
Abstract: A common approach to measuring bias in Large Language Models is to compare the log-likelihoods of two contrastive stereotype sentences. We argue that s...
By Nataliya Stepanova, Ivan Titov, Emily Allaway, Bj\"orn Ross
The paper presents an instruction‑tuned large language model (LLM) based on Qwen3 that is fine‑tuned for hate speech mitigation by unifying 36 English hate speech datasets. The authors show that this generalist LLM achieves state‑of‑the‑art performance on in‑domain benchmarks and delivers significant gains in cross‑domain and cross‑lingual generalization, outperforming specialist encoder‑based classifiers.
By Lukas Edman, Daryna Dementieva, Alexander Fraser
The paper introduces a German-English benchmark dataset to evaluate anti‑LGBTQ biases in language models, combining community‑sourced stereotypes from German‑speaking queer individuals with a German translation of WinoQueer. Eight language models of varying sizes and architectures were assessed, revealing that they reproduce anti‑queer stereotypes with differences across identities and models. Fine‑tuning on community and progressive media content reduced bias on average, though the effect was not consistent across all models and identities.
By Melina Morch, Daniel Braun
The paper introduces WinoQueer-NL, a Dutch adaptation of the English WinoQueer benchmark, designed to assess anti‑queer bias in Dutch language models. After validating the dataset with 43 queer Dutch participants, the authors expanded it to 42,906 sentences and evaluated several Dutch and multilingual models, finding that while overall bias scores appeared neutral, specific identities—particularly transgender and non‑binary—were disproportionately favored in stereotypical sentences. The study underscores the need for culturally grounded datasets to identify and mitigate biases that affect marginalized groups in Dutch NLP systems.
By Jiska Beuk, Gerasimos Spanakis
The paper introduces WinoQueer‑NL, a Dutch adaptation of the English WinoQueer benchmark, designed to assess anti‑queer bias in Dutch language models. After validating the dataset with 43 queer Dutch participants, the authors released 42,906 sentences and evaluated several Dutch‑specific and multilingual models, finding that while overall bias scores were neutral, certain models disproportionately favored stereotypical statements for transgender and non‑binary identities. The study underscores the need for culturally grounded datasets to identify and mitigate biases that affect marginalized groups in Dutch NLP systems.
arXiv:2608. 15600v1 Announce Type: new Abstract: The widespread circulation of abusive online content has increased the need for reliable moderation of Chinese social-media text.
By Mingyu Yuan, Shengtao Wen, Lingbing Guo, Zhen Bi, Xiang Chen
Melange is a synthetic visual bias benchmark designed to evaluate social biases in multi‑modal large language models (MLLMs). It extends the language‑only BBQ dataset by generating controlled visual scenarios that clearly depict target attributes, and validates images through automated VQA screening and human inspection. Using Melange, 23 open‑source MLLMs were tested in text‑only, image‑only, and image‑with‑context settings, revealing that switching from text to vision reduces fairness by an average of 0.291 and that models often choose stereotypical responses when biased answers are made.
By Razieh Rezaei, Ehsaneddin Asgary, Masoud Jalili Sabet
arXiv:2606. 27717v1 Announce Type: cross Abstract: Prosodic emphasis varies across languages, emotions, and speaking styles, yet existing emphasis detection models are largely trained and evaluated on monolingual neutral read speech.
By Megan Wei, Deepali Aneja, Jiaqi Su, Yunyun Wang, Haonan Chen, Zeyu Jin
arXiv:2608. 10810v1 Announce Type: cross Abstract: Emotion understanding in discourse requires reasoning beyond surface sentiment because speakers often convey affect through indirect, implicit, polite, ironic, or deliberately mismatched expressions.
By Zhenyan Zheng, Yunyao Zhang, Junxi Sheng, Junqing Yu, Zikai Song
arXiv:2608. 11816v1 Announce Type: cross Abstract: State-aligned distortion has been documented in China-origin text-based large language models (LLMs), but whether, and in what form, it arises in multimodal systems has not been systematically examined.
By Guang Yang, Fengchen Liu, Alex Wang, Homa Hosseinmardi, Amir Ghasemian