arXiv Computation and Language

Deep and shallow biases in language models

The paper introduces a bias depth score to differentiate between stable model preferences (Deep biases) and prompt‑dependent responses (Shallow biases) in large language models. By analyzing 4,442 opinion prompts across four models, it finds that only about a quarter of concentrated preferences persist after scenario reframing, indicating that most are shallow. The study shows Deep biases are more often inherited from pretraining and harder to remove through fine‑tuning or prompt‑based debiasing, highlighting the need to distinguish learned biases from prompt artifacts.

arXiv Machine Learning

Sep 3

GPTBIAS: A Comprehensive Framework for Evaluating Bias in Large Language Models

The paper introduces GPTBIAS, a framework that uses powerful large language models like GPT‑4 to evaluate bias in other LLMs. It employs specially crafted prompts called Bias Attack Instructions to probe for bias and outputs a bias score along with detailed information such as bias types, affected demographics, keywords, reasons, and improvement suggestions. Extensive experiments demonstrate the framework’s effectiveness and usability.

By Jiaxu Zhao, Meng Fang, Shirui Pan, Wenpeng Yin, Mykola Pechenizkiy

llms fine-tuning safety

arXiv AI

Jul 10

Adaptive Generation of Bias-Eliciting Questions for LLMs

arXiv:2510. 12857v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are now widely deployed in user-facing applications, reaching hundreds of millions of users worldwide.

By Robin Staab, Jasper Dekoninck, Maximilian Baader, Martin Vechev

llms benchmarks safety

arXiv Machine Learning

Jun 9

Exposing Hidden Biases in Text-to-Image Models via Automated Prompt Search

arXiv:2512. 08724v3 Announce Type: replace Abstract: Text-to-image (TTI) diffusion models have achieved remarkable visual quality, yet they have been repeatedly shown to exhibit social biases across sensitive attributes such as gender, race and age.

By Manos Plitsis, Giorgos Bouritsas, Vassilis Katsouros, Yannis Panagakis

llms diffusion benchmarks safety

arXiv AI

Jun 19

DeFrame: Debiasing Large Language Models Against Framing Effects

arXiv:2602. 04306v2 Announce Type: replace-cross Abstract: As large language models (LLMs) are increasingly deployed in real-world applications, ensuring their fair responses across demographics has become crucial.

By Kahee Lim, Soyeon Kim, Steven Euijong Whang

llms benchmarks safety

arXiv AI

Jun 2

One Bias After Another: Mechanistic Reward Shaping and Persistent Biases in Language Reward Models

arXiv:2603. 03291v2 Announce Type: replace-cross Abstract: Reward Models (RMs) are crucial for online alignment of language models (LMs) with human preferences.

By Daniel Fein, Max Lamparth, Violet Xiang, Mykel J. Kochenderfer, Nick Haber

llms benchmarks safety

arXiv Computation and Language

Sep 1

Large Language Models Systematically Favor Popular Options: Evidence and Mitigation Across MCQs

arXiv:2608.29257v1 Announce Type: new Abstract: Multiple-choice questions (MCQs) are a standard format for evaluating large language models (LLMs), yet the popularity of answer options can confound e...

By Abdelrahman Abdallah, Mohammed Ali, Bhawna Piryani, Mahmoud Abdalla, Adam Jatowt

llms fine-tuning benchmarks safety

arXiv AI

Sep 2

BiasGym: A Simple and Generalizable Framework for Analyzing and Removing Biases through Injection

BiasGym is a cost‑effective, generalizable framework that injects specific biases into large language models via token‑based fine‑tuning while keeping the model frozen. It then uses two debiasing methods—Scope and Steer—to identify and suppress or redirect the components responsible for biased behavior. The framework enables consistent bias elicitation, precise localization of bias associations, and targeted debiasing without harming downstream performance, and it has been shown to reduce real‑world stereotypes such as labeling Italians as reckless drivers.

By Sekh Mainul Islam, Nadav Borenstein, Siddhesh Milind Pawar, Haeun Yu, Arnav Arora, Isabelle Augenstein

llms fine-tuning safety

arXiv Machine Learning

Jun 15

Aligned but Stereotypical? How System Prompts Shape Demographic Bias in LLM-Based Text-to-Image Models

arXiv:2512. 04981v2 Announce Type: replace-cross Abstract: Text-to-image (T2I) systems increasingly rely on Large Language Model (LLM)-based text conditioning to interpret and expand user prompts.

By NaHyeon Park, Na Min An, Kunhee Kim, Soyeon Yoon, Jiahao Huo, Hyunjung Shim

llms rag diffusion benchmarks safety

arXiv AI

Jun 9

Emergence of Context Characteristics Sensitivity in Large Language Models

arXiv:2606. 09525v1 Announce Type: cross Abstract: During instruction fine-tuning (IFT), large language models (LLMs) learn to follow instructions by using the provided context to answer a query.

By Nadya Yuki Wangsajaya, Haeun Yu, Isabelle Augenstein

llms reinforcement-learning fine-tuning

arXiv AI

Aug 7

Layer-wise Positional Bias in Short-Context Language Modeling

arXiv:2601. 04098v2 Announce Type: replace-cross Abstract: Transformer language models systematically prefer tokens at specific input positions regardless of semantic relevance---a phenomenon known as positional bias.

By Maryam Rahimi, Mahdi Nouri, Yadollah Yaghoobzadeh

llms safety

arXiv AI

Aug 18

Inference-Time Mitigation of Adversarial Political Bias in Large Language Models

arXiv:2608. 14629v1 Announce Type: cross Abstract: As Large Language Models (LLMs) become the mainstay for information retrieval and summarization tasks, ensuring that they are always non-partisan and invulnerable to political bias is a critical step towards safer and more trustworthy Artificial Intelligence (AI).

By Tejaswi V. Panchagnula, Bruce Coburn, Bryce J. Dietrich, Robert X. Browning, Edward J. Delp, Fengqing Zhu

llms nlp reinforcement-learning fine-tuning safety

arXiv AI

Aug 5

Emulate or Estimate? The Divergent Strengths of Base and Post-Trained Language Models for Opinion Simulation

arXiv:2608. 03044v1 Announce Type: cross Abstract: Large language models are increasingly used to simulate human opinions, but prior work reports conflicting results: some studies find promising alignment with human survey data, while others find persona collapse and weak demographic sensitivity.

By Seth Grief-Albert, Jessica Bo, Difan Jiao, Ashton Anderson

llms safety

Related stories