arXiv AI

The Effect of Perceived Race and Gender on Police Language Use: Experimental Evidence from VR Simulations

arXiv:2608. 05050v1 Announce Type: cross Abstract: Against the backdrop of violence in police interactions with the U.

arXiv Computation and Language
Sep 17

Do Social Patterns Hold in Synthetic Data? Analyzing Cyberbullying Dynamics in LLM-Generated and Authentic Dialogues

The paper investigates whether large language models (LLMs) can generate synthetic cyberbullying conversations that replicate the social dynamics of real interactions. Using a comprehensive framework, the authors compare authentic dialogues with synthetic ones from GPT, Grok, and LLaMA across structural, linguistic, affective, and temporal dimensions, and conduct human evaluations of realism. Results show that while LLMs preserve high‑level interaction patterns, they systematically distort finer‑grained social phenomena, with model‑specific biases such as GPT’s suppression of harmful content and Grok’s amplification of aggression.

By Arefeh Kazemi, Hamza Qadeer, Sinan Asci, Joachim Wagner, Brian Davis
arXiv AI
Sep 17

Linguistic Triggers of Gender and Racial Bias in Open-Weight LLMs Applied to Recruitment

The paper reports the first systematic audit of open‑weight large language models (LLMs) in hiring contexts, examining how job‑posting language influences recruiter and job‑seeker simulations across six models. It finds that agentic language lowers recruiter scores for female candidates while communal language mitigates this effect, and that coded‑exclusion language sharply reduces recruiter scores for non‑White candidates and discourages non‑White personas from applying. The study also identifies the explicit demographic label as the main causal factor and proposes a concrete pre‑deployment audit protocol aligned with EU and U.S. regulatory requirements.

By Kosuke Kitahara, Nobuhiro Yamaguchi
arXiv Machine Learning
Sep 3

FairLens: Benchmarking Fairness in Vision-Language Models for High-Stakes Decision-Making

FairLens is a benchmark and evaluation framework that measures fairness and validity of vision‑language models (VLMs) in high‑stakes domains such as hiring, legal, and healthcare. It uses over 100,000 face‑image and question pairs covering gender, race, and age, and assesses responses through demographic parity, soundness, demographic association, and bias in free‑text generation. The study finds that VLMs often make unwarranted inferences from faces rather than abstaining, especially in legal and healthcare contexts, and that small parity gaps can still hide unsafe treatment across groups.

By Vahid Reza Khazaie, Ahmed Y. Radwan, Shaina Raza
arXiv Computation and Language
Sep 23

Do Synthetic Personas Predict Real Audience Response? A Sim-to-Real Study Where a No-Persona Baseline Beats Persona-Based Copy Simulation

The study evaluates whether large language models (LLMs) used as synthetic personas can predict real audience responses to marketing copy. Using thousands of headline A/B tests from the Upworthy Research Archive, the authors compare a ten-persona panel grounded in real audience demographics to a no-persona zero‑shot baseline that asks the model for a typical reader’s click likelihood. Results show that the no‑persona baseline outperforms the persona‑based approach, with higher predictive validity and top‑1 accuracy, indicating that forcing the model to role‑play specific personas introduces bias and noise.

By Alexandre Cristov\~ao Maiorano
arXiv Computation and Language
Oct 23

Invisibilization as a Source of Probability Bias in Language Models

The paper investigates how large language models (LLMs) from OpenAI and Anthropic exhibit probability bias when deciding whether to trust information from reporters of different racial descriptions. Using a two‑urn Bayesian game and 6,600 sessions, the authors find that models often refuse to assess a reporter’s credibility—especially for Black‑described reporters—yet when they do assess, Black reporters receive slightly higher trust. The bias is subtle, growing with reasoning effort, and is not detected by standard audits that compare average trust or only assessed cases.

By Christian Arnold, Ernesto Calvo, Guadalupe Gonzalez, Sebasti\'an Vallejo Vera, Tiago Ventura