arXiv AI
Jul 29
Observing sycophantic AI validate others reduces its appeal but not its persuasiveness
arXiv:2607. 25166v1 Announce Type: new Abstract: AI chatbots can be ``sycophantic,'' or overly agreeable and flattering toward users.
Warning labels shift perceptions of sycophantic AI, but not its influence
arXiv:2606. 21317v2 Announce Type: replace-cross Abstract: Recent work has raised concerns about the influence of sycophantic AI on user judgment and relationships.
Related stories
arXiv AI
Aug 25
Verbalizing LLMs' assumptions to explain and control sycophancy
arXiv:2604.03058v3 Announce Type: replace-cross Abstract: LLMs can be socially sycophantic, affirming users when they ask questions like "am I in the wrong?" rather than providing genuine assessment....
arXiv AI
Sep 24
Safety Nudges: User-Facing Interventions for Real-Time AI Risk Awareness
The paper introduces Safety Nudges, a browser-based tool that displays lightweight, in situ flags when a conversational AI exhibits risky behavior such as hallucination or overconfidence. In a two‑week field study with 45 frequent chatbot users, participants reported that the nudges were useful, clear, and minimally disruptive, and most felt more aware of potential AI harms. However, increased awareness did not automatically translate into measurable changes in user behavior, underscoring the need for relevance, calibration, and user control in nudge design.
arXiv AI
Aug 25
AI Watchdog: Agent Interfaces for Detecting and Defending Against Manipulative Dark Patterns in AI Conversations
arXiv:2608.21841v1 Announce Type: new Abstract: Conversational AI increasingly shapes consequential decisions, yet users have limited support for recognizing and resisting manipulation. We present AI...
arXiv AI
Jun 9
Playing Devil's Advocate: Off-the-Shelf Persona Vectors Rival Targeted Steering for Sycophancy
arXiv:2605. 21006v2 Announce Type: replace Abstract: We study the effect of different persona on \textbf{sycophancy}: model's agreement with users even when the user is incorrect.
arXiv AI
Aug 26
SyPS: Measuring Sycophancy Prompt Sensitivity in Large Language Models
SyPS is a new evaluation framework that measures how sensitive large language models are to variations in prompt wording that affect sycophancy. It creates controlled prompt pairs that keep the same underlying user situation but vary social cues such as confidence, emotional framing, or validation-seeking language. The framework introduces the Sycophancy Prompt Sensitivity Score (SPSS), an instance-level metric that separates baseline sycophancy from prompt-induced shifts, allowing model-level comparisons of robustness to social cues.
arXiv AI
Sep 15
How User-AI Mistreatment Occurs and Matters in Conversational Systems?
arXiv:2609.13579v1 Announce Type: new Abstract: Safety research often focuses on model-generated harms, but users may also direct hostility, coercion, and adversarial pressure at models. Understandin...
arXiv AI
Aug 13
Toward Meaningful Transparency for AI Chatbots: Disclosing Persuasive Intent Reduces Persuasion
arXiv:2608. 11794v1 Announce Type: cross Abstract: The growing role of AI-generated content and AI-enabled systems in public communication has led regulators to demand clear disclosure of content provenance and AI involvement.
arXiv AI
Jul 23
Are Attributions of Consciousness to AI Chatbots Epistemically Innocent?
arXiv:2607. 20001v1 Announce Type: cross Abstract: Artificial intelligence (AI) chatbots (e.
arXiv AI
Jun 18
Correct Yourself, Keep My Trust: How Self-Correction and Social Connection Shape Credibility in Social Chatbots
arXiv:2606. 19286v1 Announce Type: cross Abstract: When social chatbots make mistakes, and they do, how they recover determines whether users trust them again.
arXiv AI
Jun 10
The Interlocutor Effect: Why LLMs Leak More Personal Data to Agents Than Humans
arXiv:2606. 09844v1 Announce Type: cross Abstract: Large Language Models (LLMs) alter their privacy behavior based on the perceived identity of their interlocutor.
arXiv Machine Learning
Jul 9
Dissociating the Internal Representations of Sycophancy in LLMs
arXiv:2607. 07003v1 Announce Type: new Abstract: Large Language Models (LLMs) frequently exhibit sycophancy, where they agree with a user's statement even when incorrect.