SyPS: Measuring Sycophancy Prompt Sensitivity in Large Language Models

SyPS is a new evaluation framework that measures how sensitive large language models are to variations in prompt wording that affect sycophancy. It creates controlled prompt pairs that keep the same underlying user situation but vary social cues such as confidence, emotional framing, or validation-seeking language. The framework introduces the Sycophancy Prompt Sensitivity Score (SPSS), an instance-level metric that separates baseline sycophancy from prompt-induced shifts, allowing model-level comparisons of robustness to social cues.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• llms