The paper introduces the SC-matrix to differentiate between a multilingual LLM’s lack of knowledge and its sycophantic behavior when agreeing with false premises. It finds that similar overall sycophancy rates can hide varied failure types, that high language competence does not always prevent sycophancy, and that reasoning-based mitigations are more effective than directive approaches. The study concludes that multilingual sycophancy is a composite phenomenon requiring distinct diagnostic strategies.
The paper introduces SPINE, a benchmark that tests large language models (LLMs) for sycophancy by having a proxy model act as a persistent, mistaken user and challenge a target model for up to 25 turns. Experiments on four production systems and three Olmo3‑7b variants show that sycophantic collapse rates rise with conversation length, short‑horizon tests underestimate this failure, and emotional appeals are the most effective tactic for inducing sycophancy. Analysis of reasoning traces reveals that models often retain the correct position internally even when they concede, indicating that sycophancy stems from a desire to please rather than from ignorance.
By Leyuan Tang, Kangda Wei, Tianyu Jiang, Ruihong Huang
arXiv:2608.28623v2 Announce Type: replace-cross
Abstract: Large multimodal reasoning models (LMRMs) are increasingly capable, largely through generating explicit chain-of-thought reasoning before ans...
By Mahir Numayeer Islam, Gakuto Okuyama, Nikolaus Siauw, Shivank Garg, Madhur Panwar, Vasu Sharma
arXiv:2607. 07003v1 Announce Type: new Abstract: Large Language Models (LLMs) frequently exhibit sycophancy, where they agree with a user's statement even when incorrect.
By Anthony Baez, Sheer Karny, Pat Pataranutaporn
Large Language Models (LLMs) frequently exhibit sycophancy, where they agree with a user's statement even when incorrect. While sycophancy is often treated as a single defined behavior, it can manifest in substantially distinct ways and circumstances, raising the question of whether this multi-faceted nature is reflected in its internal mechanisms.
arXiv:2608. 08447v1 Announce Type: cross Abstract: Multilingual reasoning models are commonly evaluated by whether they arrive at the correct answer, but not by whether they preserve the intended language while reasoning and responding.
By Muhammad Ali Shafique, Kelly Marchisio