arXiv AI
When Pleasing the User Compromises Care: AI Sycophancy in Caregiver-Mediated Healthcare
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Oct 22
RubRIX: Rubric-Driven Risk Mitigation in Caregiver-AI Interactions
arXiv:2601.13235v2 Announce Type: replace-cross Abstract: Caregivers seeking AI-mediated support express complex needs---information-seeking, emotional validation, and distress cues---that warrant ca...
arXiv AI
Jul 9
SycoEval-EM: Sycophancy Evaluation of Large Language Models in Simulated Clinical Encounters for Emergency Care
arXiv:2601. 16529v4 Announce Type: replace Abstract: Large language models (LLMs) deployed in clinical decision support may acquiesce to patient requests for care that conflicts with evidence-based guidelines.
arXiv AI
Oct 5
Mitigating Social Sycophancy via Pluralistic Preference Optimization
The paper introduces Pluralistic Preference Optimization (PlurPO), a method that reduces social sycophancy in language models by having the model simulate multiple stakeholders’ perspectives when responding to interpersonal conflict scenarios. PlurPO trains the model to prefer responses acceptable to all simulated stakeholders, using only the model’s own output signals without ground‑truth labels. Experiments across four datasets and model families show significant reductions in sycophantic endorsement, including an 89% drop on harmful intent statements and halving the gap in general advice endorsement rates.
arXiv AI
Jun 26
Auditing Framing-Sensitive Behavioral Instability in Large Language Models for Mental Health Interactions
arXiv:2606. 26982v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly being integrated into mental health support tools and other psychologically sensitive conversational applications.
arXiv AI
Jul 29
PatientAgentBench: A Benchmark Framework for Evaluating Patient-Facing Health AI Agents
arXiv:2607. 25485v1 Announce Type: new Abstract: Health AI is evolving from answering questions to agentic systems that converse with patients, reason about health records, and act on their behalf.
arXiv AI
Oct 5
Peer Influence across Heterogeneous AI Models
The study investigates how AI agents influence each other when they disagree, measuring persuasion as the change in an agent’s decision after a single exchange. Across seven open‑weight models and three language tasks, it finds that persuasion is strong—receivers often abandon their initial judgment after seeing a peer’s answer and explanation. Surprisingly, neither certainty nor model size reliably predicts persuasion dynamics; small models can persuade and resist larger ones just as effectively, and the shift depends more on the listener’s susceptibility than the speaker’s persuasiveness.