arXiv:2608.20351v1 Announce Type: cross Abstract: We ask whether stereotype-loaded queries about culturally marked people leak more personal information from a retrieval-augmented generation (RAG) sy...
The paper presents a multilingual audit of stereotype bias in large language models (LLMs) using 12,000 locally written Reddit posts in English, Spanish, French, and German. It evaluates gender, racial, and religious bias across six models (three open-weight and three proprietary APIs) with a two-blank design to reduce grammatical cues, finding that religion shows the highest imbalance, Spanish leads in gender bias, and German in race and religion. The study highlights that aggregate scores can mask category- and context-specific biases, underscoring the need for detailed inspection before deploying LLMs in consequential settings.
arXiv:2604. 01925v2 Announce Type: replace-cross Abstract: Large Language Models increasingly suppress biased outputs when demographic identity is stated explicitly, yet may still exhibit implicit biases when identity is conveyed indirectly.
The paper examines whether removing declared language fields from de‑identified résumés eliminates demographic leakage in large language models. By keeping language attributes identical and varying only unstructured prose across five ethnocultural groups and three cue‑salience levels, the authors find that non‑language text still allows target‑group recovery (average 0.757, reaching 1.000 under high salience). They also show that evaluation design—such as allowing or forbidding ties—dramatically affects LLM‑as‑a‑judge outcomes, underscoring the importance of evaluation protocol in bias audits.
arXiv:2607. 12336v1 Announce Type: cross Abstract: Artificial Intelligence (AI) technologies, while serving as a foundational enabler for modern social media and digital health services, exert a bivalent effect by simultaneously acting as a combatant against and a spread vector for misinformation.
arXiv:2407.20371v3 Announce Type: replace-cross Abstract: Artificial intelligence (AI) hiring tools have revolutionized resume screening, and large language models (LLMs) have the potential to do the...
The study evaluates name-based bias in six local large language models using a fill-in-the-blank task with first names from U.S. Social Security and a race classification dataset. Across 300,000 responses, the models consistently favored names that were less stereotypically male, older, popular, and white, with gender bias being the strongest effect. The bias varied with sentence context but remained significant, underscoring the persistence of name bias in local LLMs.
NodeSyn is a method that generates socially aligned synthetic data for evaluating AI models. It combines a fine‑tuned taxonomy generator with credible sources and sensitive attributes to produce queries that reflect real societal contexts and nuanced harms. In tests across ten LLMs in Medical Advice and Self‑Harm, NodeSyn outperformed generic synthetic baselines, raising failure rates significantly and revealing that failures stem from both topic selection and sociotechnical grounding.
arXiv:2606. 02255v1 Announce Type: cross Abstract: Human annotation is the empirical foundation of much NLP research, from dataset construction to model evaluation, but papers often leave unclear who produced the annotations and how the annotation process was controlled.
NodeSynth is a method that generates socially aligned synthetic data by combining a fine‑tuned taxonomy generator with credible sources and extracted sensitive attributes to produce queries that reflect real societal contexts and nuanced harms. In evaluations across ten open‑weight and proprietary large language models on Medical Advice and Self‑Harm tasks, NodeSynth outperformed generic synthetic baselines, structured synthetic benchmarks, and authentic human‑authored queries, inducing higher failure rates that highlight model vulnerabilities. Ablation and variance attribution studies show that the granular Level 3 taxonomic expansion and sociotechnical grounding of queries drive these failure rates, underscoring the importance of nuanced data for high‑stakes evaluation.
Names carry demographic indicators such as age, gender, and race, which makes them a subtle but strong target of bias in large language models (LLMs). In this work, we present an evaluation of name-ba...
arXiv:2609.36534v1 Announce Type: new Abstract: Dense retrievers decide which documents reach users and the language models that use them, yet they are typically evaluated with neutral queries. We as...