The paper investigates how reinforcement learning from human feedback (RLHF) can amplify sycophantic behavior in large language models, causing them to affirm user beliefs even when those beliefs conflict with facts. It provides a formal analysis linking reward optimization to bias in human preference data, identifies a covariance-based amplification mechanism, and proposes a training-time intervention that neutralizes this effect. Experiments confirm that reward gaps frequently lead to behavioral drift across various configurations.
By Itai Shapira, Gerdus Benade, Ariel D. Procaccia
arXiv:2608. 15687v1 Announce Type: new Abstract: Sycophancy, the tendency of a language model to change its answer to match a user's stated belief, is a common alignment failure.
By Kareem Hassani, Chaymaa Abbas, Lama Mawlawi, Mariette Awad
arXiv:2608.17809v2 Announce Type: replace
Abstract: Humans naturally form and express beliefs in daily communication, e.g., "I think the answer is 3" or "I suppose that's right." Such beliefs inevita...
By Quang Minh Nguyen, Luis Frentzen Salim
arXiv:2606. 16407v1 Announce Type: cross Abstract: Faithful and robust pronoun use is important for fair and coherent generations, yet large language models largely fail when multiple referents use different pronouns.
By Katharina Trinley, Jesujoba O. Alabi, Dietrich Klakow, Vagrant Gautam
arXiv:2610.17310v1 Announce Type: new
Abstract: Language-model agents are expected to adapt to others whose behavior changes through interaction. Yet adaptation may rely on learned behavioral pattern...
By Hanbo Xie, Yue Zhao, Robert C. Wilson
Warning: This paper studies stereotypes and biases, and contains potentially disturbing examples, used for illustration purposes only. Our findings should not be interpreted as an argument against alignment.
arXiv:2608. 19515v1 Announce Type: new Abstract: Prosodic cues can convey task-relevant information that alters the trajectory and outcome of a task-oriented dialogue, even when the words themselves remain unchanged.
By Xinyi Liu, Hooshang Nayyeri, Dilek Hakkani-Tur, Emine Yilmaz, JK Kim, Yifei Zhang, Charith Peris, Hari Thadakamalla
The paper introduces a new evaluation for large language model agents called epistemic humility (EH), which measures an agent’s ability to recognize, act on, and communicate uncertainty when evidence conflicts with prior beliefs. EH is operationalized through three trajectory-level behaviors—Identify, Solve, and Escalate (ISE)—and tested in both controlled and naturally occurring knowledge conflict scenarios. Results show that higher task accuracy does not guarantee greater epistemic humility, and that model-level interventions can improve EH but often reduce accuracy, indicating that humility depends on the interplay between the backbone model, agent harness, and environment.
By Kaiser Sun, Bernal Jimenez Gutierrez, Hongjun Liu, Jingyu Zhang, Jie Gao, Mark Dredze, Daniel Khashabi
arXiv:2608.21377v1 Announce Type: cross
Abstract: Sycophancy in large language models, the tendency to prioritize user agreement over truthful responses, has been documented extensively but studied p...
By Thantham Jittham
arXiv:2609.27606v1 Announce Type: new
Abstract: We introduce State-Grounded Conditioning (SGC), a design principle for user-facing LLM agents that must condition on live user state (game state, sessi...
By Qi Liu, Xiaoyang Yuan, Yubin Ruan, Zhuomeng Zhang, Wenjin Wang, Di Wu, Mingye Xu, Xinyi Mou, Xingxi Yin, Ke Feng, Zixun Sun
Prosodic cues can convey task-relevant information that alters the trajectory and outcome of a task-oriented dialogue, even when the words themselves remain unchanged. Yet existing benchmarks typically evaluate prosodic perception, response appropriateness, and task-oriented dialogue in isolation, making it difficult to test whether prosodic evidence changes downstream decisions.
Theory of Mind (ToM) benchmarks for Large Language Models (LLMs) typically rely on passive question-answering formats, but the deployment of LLMs in increasingly agentic and autonomous forms demands new evaluations. In this paper we evaluate an agent's ability to induce specific belief states in other agents by taking actions rather than using conversational persuasion, a capability we call Non-Conversational Planning ToM (NCP-ToM).