arXiv AI
The Role of Implicit and Explicit Demographic Signals in Large Language Model-based Student Assessment
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Computation and Language
1d ago
Which Demographics do LLMs Default to During Annotation?
arXiv:2410.08820v4 Announce Type: replace Abstract: Demographics and cultural background of annotators influence the labels they assign in text annotation -- for instance, an elderly woman might find...
arXiv Computation and Language
Sep 2
LLM-as-a-Demographic: Whom Sociodemographic Prompting Helps, and Whom It Hurts
arXiv:2609.00222v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used as judges for subjective tasks, where annotators disagree and the relevant question is not only how...
Hugging Face Trending Papers
Aug 31
LLM-as-a-Demographic: Whom Sociodemographic Prompting Helps, and Whom It Hurts
Large language models (LLMs) are increasingly used as judges for subjective tasks, where annotators disagree and the relevant question is not only how accurate a judge is, but whose judgments it repro...
arXiv AI
Jul 10
Persona Matters: Effects of Activation Steering on Short Answer Generation and Scoring
arXiv:2604. 07102v2 Announce Type: replace-cross Abstract: Activation-based steering enables inference-time personalization of large language models, but its effects in educational applications are not well understood.
arXiv AI
Jul 24
ImplicitBBQ: Benchmarking Implicit Bias in Large Language Models through Characteristic Based Cues
arXiv:2604. 01925v2 Announce Type: replace-cross Abstract: Large Language Models increasingly suppress biased outputs when demographic identity is stated explicitly, yet may still exhibit implicit biases when identity is conveyed indirectly.
arXiv Computation and Language
Sep 1
Different Demographic Cues Yield Inconsistent Conclusions About LLM Personalization and Bias
The paper examines how large language models (LLMs) respond to different demographic cues—such as names—when users seek advice, focusing on race and gender in a U.S. context. It finds that using different cues for the same group leads to only partially overlapping changes in model responses, producing inconsistent conclusions about personalization and unstable bias metrics. The authors argue that LLMs react to linguistic signals tied to cues rather than to stable demographic categories, and they call for evaluations that use multiple cues and consider underlying mechanisms.