arXiv Computer Vision

Guardrail-Agnostic Societal Bias Evaluation in Large Vision-Language Models

arXiv Computation and Language
1d ago

Evaluating Second-Order Bias of LLMs Through Epistemic Entitlement

The paper introduces a novel framework for assessing second‑order bias in large language models (LLMs), defined as bias in how an LLM judges the acceptability of biased content. Using principles from entitlement epistemology, the authors design a reasoning task that asks LLMs to determine whether a biased text is acceptable for specific demographic groups, and propose two metrics to quantify biased judgments. Experiments on both open‑source and closed‑source models reveal that the task bypasses safety guardrails, uncovers systematic variations across target groups, and demonstrates that models still rely on demographic labels when evaluating bias.

By Ramaravind Kommiya Mothilal, Terry Jingchen Zhang, Raiyan Ahmed, Zhijing Jin, Shion Guha, Syed Ishtiaque Ahmed
arXiv AI
Aug 24

Who Do Language Models Think Is Competent? A Mechanistic Analysis of Occupational Bias

The paper investigates whether language models still encode occupational biases even when they appear unbiased in behavioral tests. Using a causal framework, the authors separate bias into internal representations of user competence and observable outputs, deriving steering vectors that show these representations influence model behavior in question‑answering and hiring tasks. Across several open‑weight models, demographic factors such as gender, race, and socioeconomic status affect the models’ internal competence representations, revealing hidden bias that behavioral metrics alone may miss.

By Keren Fuentes, Aaron Mueller
arXiv Computer Vision
2d ago

Frontier vision-language models have overtaken young adults at detecting AI-generated portraits -- but not their calibration

arXiv:2608.30210v1 Announce Type: cross Abstract: AI image generators now create face portraits that are hard to tell from real photographs. Vision-language models (VLMs) are increasingly proposed to...

By Sunwhi Kim (Hwasung Medi-Science University, Dept. of Bio-Healthcare), Sunyul Kim (Yonsei University, Graduate School of Engineering, Dept. of Artificial Intelligence), Meounggun Jo (Hoseo University), Jini Tae (Gwangju Institute of Science and Technology, School of Humanities and Social Sciences)