arXiv AI
3d ago

Persona Following Is Not Selective Control: The Neutrality Gap in LLM User Simulation

Persona prompting is widely used in LLM user simulation, yet it assumes that specifying a target user attribute will shift only that attribute while leaving others unchanged. Across multiple models, the study finds that changing a target attribute also alters unspecified non‑target choices, creating a "neutrality gap" that persists even when no preference is declared. The authors show that blocking query‑time attention reduces non‑target drift, but this also weakens target efficacy, highlighting a central challenge in achieving selective control.

By Jiashen Ren, Wenlin Zhang, Bohan Zhang, Xiaopeng Li, Zichuan Fu, Wanyu Wang, Junyi Li, Xiangyu Zhao
arXiv Computation and Language
Nov 4

Swap, Replicate, Read the Mass: A Protocol for Auditing Demographic Bias in System One Decision Models

The paper introduces a comprehensive audit protocol for System One decision models, which provide typed, probability‑scored answers without human oversight. Using three adapted benchmarks—BBQ, CrowS‑Pairs, and SilverSpoon—it evaluates twelve models plus a no‑thinking baseline across 1,638,000 decisions, revealing widespread stereotype alignment, order‑dependent responses, and confidence instability. The study demonstrates that swapping answer options and reversing question order significantly alters model outputs, underscoring the need for order‑invariant scoring in procurement.

By Alejandro Mozo Quesada (Universidad de M\'alaga, Spain), Sergio G\'alvez Rojas (Universidad de M\'alaga, Spain), Francisco L\'opez Valverde (Universidad de M\'alaga, Spain)