arXiv Computation and Language
Initial Evaluation of Potential Bias in Coverage of Humans in Wikidata
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv Machine Learning
Aug 24
Exploratory As-Analyzed No-Detection of Culturally-Marked Predicate-Triggered PII Amplification in a Synthetic-English RAG Probe: A Predicate-Resource-Confounded Audit
arXiv:2608.20351v1 Announce Type: cross Abstract: We ask whether stereotype-loaded queries about culturally marked people leak more personal information from a retrieval-augmented generation (RAG) sy...
arXiv Computation and Language
2d ago
From Language to Country to Valence: A Multilingual Audit of Stereotype Bias Across LLMs
The paper presents a multilingual audit of stereotype bias in large language models (LLMs) using 12,000 locally written Reddit posts in English, Spanish, French, and German. It evaluates gender, racial, and religious bias across six models (three open-weight and three proprietary APIs) with a two-blank design to reduce grammatical cues, finding that religion shows the highest imbalance, Spanish leads in gender bias, and German in race and religion. The study highlights that aggregate scores can mask category- and context-specific biases, underscoring the need for detailed inspection before deploying LLMs in consequential settings.
arXiv AI
Jul 24
ImplicitBBQ: Benchmarking Implicit Bias in Large Language Models through Characteristic Based Cues
arXiv:2604. 01925v2 Announce Type: replace-cross Abstract: Large Language Models increasingly suppress biased outputs when demographic identity is stated explicitly, yet may still exhibit implicit biases when identity is conveyed indirectly.
arXiv AI
Sep 16
Beyond the Name: Demographic Leakage in De-Identified R\'esum\'es and Evaluation Artifacts in LLM Bias Audits
The paper examines whether removing declared language fields from de‑identified résumés eliminates demographic leakage in large language models. By keeping language attributes identical and varying only unstructured prose across five ethnocultural groups and three cue‑salience levels, the authors find that non‑language text still allows target‑group recovery (average 0.757, reaching 1.000 under high salience). They also show that evaluation design—such as allowing or forbidding ties—dramatically affects LLM‑as‑a‑judge outcomes, underscoring the importance of evaluation protocol in bias audits.
arXiv AI
Jul 15
Evaluating Health Misinformation in Low-Resource Languages: Integrating Small Language Models with a Culturally-Sensitive Responsible NLP Framework (Bangla as a Case Study)
arXiv:2607. 12336v1 Announce Type: cross Abstract: Artificial Intelligence (AI) technologies, while serving as a foundational enabler for modern social media and digital health services, exert a bivalent effect by simultaneously acting as a combatant against and a spread vector for misinformation.
arXiv AI
Sep 1
Gender, Race, and Intersectional Bias in Resume Screening via Language Model Retrieval
arXiv:2407.20371v3 Announce Type: replace-cross Abstract: Artificial intelligence (AI) hiring tools have revolutionized resume screening, and large language models (LLMs) have the potential to do the...