arXiv AI
How Humans and LLMs Read Gender into Gender-Neutral Physical Descriptions
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Sep 18
How Humans and LLMs Read Gender into "Gender-Neutral" Physical Descriptions
The study introduces GAPA, a dataset of 316 physical attributes with 14,706 gender-association ratings from 304 US annotators, showing that such descriptions carry structured gender associations. It evaluates 16 LLMs, finding they partially mirror human ratings but exhibit biases such as compressed distributions, weaker alignment for men, and asymmetric abstention toward non‑binary identities. A proxy model trained on these data is released and applied to analyze character descriptions in LitBank, illustrating the persistence of gendered interpretations in ostensibly neutral language.
arXiv AI
Sep 1
Vision-Language Models Suppress Female Representations Under Ambiguous Input
arXiv:2605.31556v2 Announce Type: replace-cross Abstract: Alignment teaches vision-language models (VLMs) to avoid expressing demographic biases, and when gender is clearly visible they largely succe...
arXiv AI
Jun 18
When Cars Have Stereotypes: Auditing Demographic Bias in Objects from Text-to-Image Models
arXiv:2508. 03483v3 Announce Type: replace-cross Abstract: While prior research on text-to-image generation has predominantly focused on biases in human depictions, demographic bias in generated objects remains relatively underexplored.
arXiv Computation and Language
Sep 11
Alignment Reduces Expressed but Not Encoded Gender Bias: A Unified Framework and Study
The paper introduces a unified framework that simultaneously measures intrinsic (encoded) and extrinsic (expressed) gender bias in large language models using identical neutral prompts. It finds a consistent link between latent gender information and output bias, but shows that alignment via supervised fine‑tuning reduces expressed bias while leaving internal gender associations largely intact and reactivatable by adversarial prompts. The study also demonstrates that debiasing gains on structured benchmarks may not transfer to realistic tasks such as story generation.
arXiv Computation and Language
Sep 17
Which Demographics do LLMs Default to During Annotation?
The paper investigates which demographic attributes large language models (LLMs) default to when annotating text without explicit demographic cues. By comparing non‑demographic, placebo‑conditioned, and demographic‑conditioned prompts on politeness and offensiveness tasks in the POPQUORN dataset, the authors find that LLMs exhibit notable gender, race, and age influences in their annotations. This contrasts with earlier studies that reported no such effects, highlighting the importance of considering demographic bias in LLM‑based annotation workflows.
arXiv Computation and Language
1d ago
LLMs Cannot Be Neutral: Gender Sensitivity Across Candidates, Evaluators, and Models
The study investigates how gender assignments of candidates and evaluators influence decisions made by large language models (LLMs) in a city council voting scenario. Eight LLM-as-a-Judge and decision language models evaluated 240 synthetic professional narratives, producing 30,720 judgments. Results show that explicit candidate gender labels slightly increased the probability of a Yes vote, only nonbinary evaluator assignments had a significant effect, and decision reversals varied across models, with decision language models exhibiting fewer reversals than LLM-as-a-Judge models.