arXiv AI

A Multifaceted Analysis of Social Biases in Large Language Models

arXiv:2512. 15792v4 Announce Type: replace-cross Abstract: Large language models (LLMs) have rapidly become indispensable tools for acquiring information and supporting human decision-making.

arXiv AI
Sep 18

Geopolitical Divisions Across Languages in Large Language Models

The study examines how language influences AI chatbot responses to questions about the war in Ukraine, revealing that the same AI systems (GPT, Claude, Gemini) produce varying political stances across 112 languages. By evaluating 20 statements in 112 languages, the researchers found that Russia‑leaning versus Ukraine‑leaning answers differ by language, mirroring global political attitudes such as public support for Russia, UN voting patterns, and aid levels. This pattern persists across all three models and even when specific statement pairs are removed, suggesting that information warfare could embed geopolitical biases into AI training data.

By Maxim Chupilkin
arXiv AI
Jun 30

LLM-Ideoplasticity: Measuring Ideological Plasticity in the Political Behavior of LLMs as a Context-Conditioned Distribution

arXiv:2606. 28335v1 Announce Type: cross Abstract: We argue, with systematic empirical evidence, that a large language model's political ideology is not a fixed point, but a conditional distribution $\mathbb{P}($position$\mid$context$)$ over a real political space.

By Adib Sakhawat, Syed Rifat Raiyan, Tahsin Islam, Takia Farhin, Hasan Mahmud, Md Kamrul Hasan
arXiv Machine Learning
Sep 3

GPTBIAS: A Comprehensive Framework for Evaluating Bias in Large Language Models

The paper introduces GPTBIAS, a framework that uses powerful large language models like GPT‑4 to evaluate bias in other LLMs. It employs specially crafted prompts called Bias Attack Instructions to probe for bias and outputs a bias score along with detailed information such as bias types, affected demographics, keywords, reasons, and improvement suggestions. Extensive experiments demonstrate the framework’s effectiveness and usability.

By Jiaxu Zhao, Meng Fang, Shirui Pan, Wenpeng Yin, Mykola Pechenizkiy
arXiv Computation and Language
Sep 1

Political Ideology Shifts in Large Language Models

arXiv:2508.16013v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly deployed in politically sensitive contexts, raising concerns about their susceptibility to ideologica...

By Pietro Bernardelle, Stefano Civelli, Leon Fr\"ohling, Riccardo Lunardi, Kevin Roitero, Gianluca Demartini
arXiv AI
Sep 1

Evaluating and Mitigating Anti-LGBTQ Biases in German and Multilingual Language Models

The paper introduces a German-English benchmark dataset to evaluate anti‑LGBTQ biases in language models, combining community‑sourced stereotypes from German‑speaking queer individuals with a German translation of WinoQueer. Eight language models of varying sizes and architectures were assessed, revealing that they reproduce anti‑queer stereotypes with differences across identities and models. Fine‑tuning on community and progressive media content reduced bias on average, though the effect was not consistent across all models and identities.

By Melina Morch, Daniel Braun