Hugging Face Trending Papers
LLM-as-a-Demographic: Whom Sociodemographic Prompting Helps, and Whom It Hurts
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv Computation and Language
Sep 2
LLM-as-a-Demographic: Whom Sociodemographic Prompting Helps, and Whom It Hurts
arXiv:2609.00222v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used as judges for subjective tasks, where annotators disagree and the relevant question is not only how...
arXiv Computation and Language
1d ago
Which Demographics do LLMs Default to During Annotation?
arXiv:2410.08820v4 Announce Type: replace Abstract: Demographics and cultural background of annotators influence the labels they assign in text annotation -- for instance, an elderly woman might find...
arXiv AI
Aug 13
Large Language Models Reproduce Racial Stereotypes When Used for Text Annotation
arXiv:2603. 13891v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly used for automated text annotation in tasks ranging from academic research to content moderation and hiring.
arXiv Computation and Language
Sep 1
Different Demographic Cues Yield Inconsistent Conclusions About LLM Personalization and Bias
The paper examines how large language models (LLMs) respond to different demographic cues—such as names—when users seek advice, focusing on race and gender in a U.S. context. It finds that using different cues for the same group leads to only partially overlapping changes in model responses, producing inconsistent conclusions about personalization and unstable bias metrics. The authors argue that LLMs react to linguistic signals tied to cues rather than to stable demographic categories, and they call for evaluations that use multiple cues and consider underlying mechanisms.
arXiv AI
2d ago
The Role of Implicit and Explicit Demographic Signals in Large Language Model-based Student Assessment
arXiv:2609.16993v1 Announce Type: cross Abstract: Large Language Models are now common in student assessment, but we know little about how student demographics affect their use. Sometimes, considerin...
arXiv Computation and Language
Sep 2
Evaluating Second-Order Bias of LLMs Through Epistemic Entitlement
The paper introduces a novel framework for assessing second‑order bias in large language models (LLMs), defined as bias in how an LLM judges the acceptability of biased content. Using principles from entitlement epistemology, the authors design a reasoning task that asks LLMs to determine whether a biased text is acceptable for specific demographic groups, and propose two metrics to quantify biased judgments. Experiments on both open‑source and closed‑source models reveal that the task bypasses safety guardrails, uncovers systematic variations across target groups, and demonstrates that models still rely on demographic labels when evaluating bias.