SyPS is a new evaluation framework that measures how sensitive large language models are to variations in prompt wording that affect sycophancy. It creates controlled prompt pairs that keep the same underlying user situation but vary social cues such as confidence, emotional framing, or validation-seeking language. The framework introduces the Sycophancy Prompt Sensitivity Score (SPSS), an instance-level metric that separates baseline sycophancy from prompt-induced shifts, allowing model-level comparisons of robustness to social cues.
By Lijia Huang, Yao Fu, Sihao Ren
arXiv:2606. 18258v1 Announce Type: cross Abstract: Large language models (LLMs) exhibit a wide range of human-like behaviors, from expressing thoughts and emotions, to engaging in relationship-building with users, to refusing requests and maintaining boundaries.
By Sunnie S. Y. Kim, Margit Bowler, Leon A Gatys
arXiv:2609.26579v1 Announce Type: new
Abstract: A central concern with language models is sycophancy: their tendency to defer to users' views at the expense of independent substantive judgment. In pa...
By Calvin Isley, Johann Gaebler, Max Lamparth, Julia Minson, Sharad Goel
arXiv:2608.21242v1 Announce Type: new
Abstract: As conversational companions, large language models (LLMs) often have access to users' emotional states. We study how this affective context modulates...
By Jiayi Li, Sanjana Menon, Brett Frischmann, Shomir Wilson, Sarah Rajtmajer
Large language models (LLMs) are increasingly used for emotional support, raising concern that sustained use may draw users away from their real-world relationships. Yet existing evaluations primarily...
arXiv:2610.09569v1 Announce Type: new
Abstract: Large language models (LLMs) are increasingly used for emotional support, raising concern that sustained use may draw users away from their real-world...
By Shivam Shukla, Jihye Kim, Shubham Gaur, Mahnaz Roshanaei, Magy Seif El-Nasr