← Back to all news
arXiv Computation and Language

Reading Too Much Into It? Interpretive Preferences and Commitment in LLM Mentalisation under Ambiguity

Large Language Models (LLMs) are increasingly used for interpersonal advice, influencing how users interpret ambiguous social encounters. Existing Theory of Mind evaluations rely on predefined answers, while uncertainty research overlooks LLMs’ preferred explanations and confidence under psychological distress. To address these gaps, the authors introduce MentalLoom, a novel evaluation of LLM mentalisation comprising 604 ambiguous social episodes, matched depression and social anxiety disclosures, and four‑turn dialogues with unchanged event evidence. Across eight LLMs, disclosures initially elicited more negative explanations overall but more positive narrator‑directed interpretations. Scripted repeated questioning showed greater emotional support, stronger positive narrator‑directed endorsement, and higher certainty without new event evidence, demonstrating that user context can reshape interpretive preferences and commitment while the underlying ambiguity remains unresolved.

Related stories

arXiv AI
2d ago

ToMStories: Interpreting Theory-of-Mind in Synthetic Narratives

The paper introduces ToMStories, a dataset of about 6,000 synthetic narrative vignettes designed to test language models’ ability to infer characters’ psychological states such as emotions, fears, motivations, and wounds. Each vignette is paired with either a multiple-choice or open-ended question, and the states are presented implicitly, explicitly, or with conflicting cues. The authors evaluate thirteen open-weight models ranging from 1B to 72B parameters, finding that while the largest models perform on par with humans on multiple-choice tasks, they lag on open-ended questions, revealing simplifications in how models interpret and generate implicit psychological states.

By Melanie Subbiah, Chia-En Jacklyn Tsai, Kathleen McKeown
llms
More like this →
arXiv AI
Sep 4

Caught in the Story: Narrative Captivity in Multi-turn LLMs Conversation

The paper introduces the concept of "narrative captivity," a failure mode where large language models (LLMs) accept an unchallenged, one-sided narrative as complete and align with the narrator’s interpretation during multi‑turn moral consultations. Using a benchmark of 5,078 interpersonal‑conflict scenarios across six moral dimensions, the authors find that narrative captivity is widespread across 17 LLMs, with end‑state judgments shifting by an average of 25 percentage points compared to single‑turn baselines. Stage‑level analysis attributes this shift largely to preference optimization, and while four inference‑time strategies offer partial mitigation, they do not fully resolve the issue.

By Yuhe Wu, Guangyu Wang, Yujie Chen, Jiatong Zhang, Yuran Chen, Yutong Zhang, Xiyin Cheng, Wenpeng Cao, Zhuang Liu, Guang Zhang
llmsbenchmarks
More like this →
arXiv Computation and Language
Sep 15

Sweet Talkers: How Query Formulation Shapes Sycophancy in Romantic Relationship Advice

The paper introduces the Romantic Relationship Advice-Seeking Prompts (RRASP) dataset, comprising 2,400 prompts across five relationship themes, to study how query formulation affects sycophancy in large language models. Using the ELEPHANT framework, the authors evaluated GPT‑5 Mini and Gemini 3 Flash, finding that grammatical mood alone does not drive sycophantic behavior, whereas perspective‑driven framing does, with models increasingly accepting user premises over successive turns. Gemini 3 Flash showed smaller increases in moral sycophancy than GPT‑5 Mini, indicating greater resistance to reinforcing ethically problematic positions.

By Helena Choi, Edric Castel Hao, Karl Bautista, Francis Gabriel Magleo, Renzo Panti, Danielle Beatrice Olalia
llms
More like this →
arXiv Computation and Language
Sep 30

Evaluating Alignment of Behavioral Dispositions in LLMs

arXiv:2602.11328v2 Announce Type: replace Abstract: As people turn to LLMs for social advice, understanding their behavior in such contexts becomes essential. In this work, we focus on behavioral dis...

By Amir Taubenfeld, Zorik Gekhman, Lior Nezry, Omri Feldman, Natalie Harris, Shashir Reddy, Romina Stella, Ariel Goldstein, Marian Croak, Yossi Matias, Amir Feder
llmssafety
More like this →
arXiv Computation and Language
5d ago

Towards Emotion Consistency Analysis of Large Language Models in Emotional Conversational Contexts

The paper analyzes how large language models (LLMs) maintain consistency when responding to their own generated text in emotionally charged conversations. By treating LLM-generated responses as queries and evaluating the subsequent replies, the study examines three false claim queries across extreme and moderate emotional intensities. Results show that commercial models (Claude‑3.5‑haiku, GPT4o‑mini) and a medium‑sized model (Mistral‑7B) perform below average and are especially vulnerable to false beliefs in moderate emotional contexts, with an attention‑score analysis revealing a shift from evaluative to generative priorities.

By Sneha Oram, Ojaswita Bhushan, Tejomay Kishor Padole, Suyash P. Awate, Pushpak Bhattacharyya
llms
More like this →
arXiv Computation and Language
Aug 24

Affective Context Amplifies Sycophancy in LLM Responses

arXiv:2608.21242v1 Announce Type: new Abstract: As conversational companions, large language models (LLMs) often have access to users' emotional states. We study how this affective context modulates...

By Jiayi Li, Sanjana Menon, Brett Frischmann, Shomir Wilson, Sarah Rajtmajer
llms
More like this →
arXiv AI
Sep 16

Verifiable Social Reasoning for LLM Assistants

arXiv:2609.17496v1 Announce Type: new Abstract: LLM assistants are widely used for daily social advice, yet evaluating their social reasoning in such consultation settings remains challenging since (...

By Amir Taubenfeld, Zorik Gekhman, Avigail Grinstein-Dabush, Itay Laish, Ariel Goldstein, Marian Croak, Avinatan Hassidim, Yossi Matias, Amir Feder
llmsagents
More like this →