arXiv AI September 17, 2026 By Chowdhury Mohammad Abdullah, Rita Orji

From a River in Gilead to the Inference Distributions of Large Language Models: Covert Dialect Bias and Linguistic Profiling at Scale

The paper investigates covert dialect bias in large language models (LLMs) by analyzing how internal probability distributions associate different English varieties—Standard American English, African American Vernacular English, Nigerian Standard English, and Nigerian Pidgin—with housing-related adjectives. Using 260 meaning‑matched sentence quadruples and log‑probability scoring across ten open‑weight LLMs, the study finds that AAVE and NP are consistently linked to more negative adjectives than SAE, with NP experiencing the greatest penalty. The bias varies by context and stereotype cluster, and Nigerian Standard English shows a context‑dependent shift, being favored in formal tenant screening but penalized in more socially proximate scenarios.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Aug 13

Large Language Models Reproduce Racial Stereotypes When Used for Text Annotation

arXiv:2603. 13891v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly used for automated text annotation in tasks ranging from academic research to content moderation and hiring.

By Petter T\"ornberg

llms fine-tuning safety

arXiv Machine Learning

Jul 24

How Robust Is Homogeneity Bias in LLMs? Evidence Across Models, Decoding Settings, and Identity Signals

arXiv:2501. 02211v3 Announce Type: replace-cross Abstract: Large language models (LLMs) reproduce homogeneity bias -- the tendency to portray marginalized groups as more internally similar than dominant groups -- but whether this bias generalizes across models, is stable under different inference settings, or depends on how group identity is signaled remains unstudied.

By Messi H. J. Lee

llms safety

arXiv AI

Jun 2

IndoBias: A Dual Track Culturally Grounded Benchmark for LLMs Bias Evaluation in Indonesian Languages

arXiv:2606. 01260v1 Announce Type: cross Abstract: Despite being home to more than 1300 ethnic groups and 700 indigenous languages, bias in Large Language Models has not been fully studied in Indonesia, thus leaving a critical gap in evaluating representational fairness and localized stereotypes within its uniquely vast, multilingual, and diverse sociocultural landscape.

By Ikhlasul Akmal Hanif, Muhammad Falensi Azmi, Filbert Aurelian Tjiaranata, Eryawan Presma Yulianrifat, Fajri Koto

llms benchmarks safety

arXiv Computation and Language

4d ago

Retrieval Sensitivity to Identity Signals in Queries

arXiv:2609.36534v1 Announce Type: new Abstract: Dense retrievers decide which documents reach users and the language models that use them, yet they are typically evaluated with neutral queries. We as...

By Andrew Tang, Nicholas Deas, Kathleen McKeown, Vishal Misra

llms rag safety

arXiv Machine Learning

Jun 25

Homogeneity Bias in Open-Weight LLMs Is Robust to Decoding Hyperparameters

arXiv:2501. 02211v2 Announce Type: replace-cross Abstract: Large language models (LLMs) reproduce homogeneity bias -- the tendency to portray marginalized groups as more internally similar than dominant groups -- but whether this bias is stable or an artifact of inference settings has only been studied in single proprietary models.

By Messi H. J. Lee

llms safety

arXiv AI

Jul 24

ImplicitBBQ: Benchmarking Implicit Bias in Large Language Models through Characteristic Based Cues

arXiv:2604. 01925v2 Announce Type: replace-cross Abstract: Large Language Models increasingly suppress biased outputs when demographic identity is stated explicitly, yet may still exhibit implicit biases when identity is conveyed indirectly.

By Bhaskara Hanuma Vedula, Darshan Anghan, Ishita Goyal, Ponnurangam Kumaraguru, Abhijnan Chakraborty

llms benchmarks safety

One email a morning, machine-written

Related stories

Large Language Models Reproduce Racial Stereotypes When Used for Text Annotation

How Robust Is Homogeneity Bias in LLMs? Evidence Across Models, Decoding Settings, and Identity Signals

IndoBias: A Dual Track Culturally Grounded Benchmark for LLMs Bias Evaluation in Indonesian Languages

Retrieval Sensitivity to Identity Signals in Queries

Homogeneity Bias in Open-Weight LLMs Is Robust to Decoding Hyperparameters

ImplicitBBQ: Benchmarking Implicit Bias in Large Language Models through Characteristic Based Cues