The paper introduces ToMStories, a dataset of about 6,000 synthetic narrative vignettes designed to test language models’ ability to infer characters’ psychological states such as emotions, fears, motivations, and wounds. Each vignette is paired with either a multiple-choice or open-ended question, and the states are presented implicitly, explicitly, or with conflicting cues. The authors evaluate thirteen open-weight models ranging from 1B to 72B parameters, finding that while the largest models perform on par with humans on multiple-choice tasks, they lag on open-ended questions, revealing simplifications in how models interpret and generate implicit psychological states.
arXiv:2603.18007v2 Announce Type: replace-cross Abstract: The study explores whether current Large Language Models (LLMs) exhibit Theory of Mind (ToM) capabilities -- specifically, the ability to inf...
The paper explores whether language models encode components that represent narrativity. By examining GPT‑2‑Small, Gemma‑2‑2B, and Llama‑3.1‑8B, the authors identify specific attention heads, MLP neurons, and sparse autoencoder features whose activations correlate with narrative properties. Ablating these components raises next‑token prediction loss more for high‑narrativity texts and diminishes narrativity in open‑ended generations, with the strongest effects seen in story completion and narrative question answering tasks.
arXiv:2502. 09487v4 Announce Type: replace-cross Abstract: Narratives and emotions shape thoughts, and thoughts shape our feelings and stories we tell.
arXiv:2605. 09159v2 Announce Type: replace Abstract: Recent work shows that large language models (LLMs) encode behavioral traits ("personas") as linear directions in activation space, often called "persona vectors".
arXiv:2412. 15239v4 Announce Type: replace-cross Abstract: Consumers' engagement with stories is shaped by their expectations about what will happen next, yet modeling these forward-looking beliefs over unstructured narrative content has remained challenging.
arXiv:2609.38406v1 Announce Type: new Abstract: Access to real-world information is often noisy and fragmented. Constructing a coherent narrative from such fragments requires models to reconstruct mi...
Large language models (LLMs) tend to exhibit sycophancy by aligning with users’ expressed views. This study introduces a benchmark of 5,500 evaluation instances built from 500 debatable questions across ten domains, where the user stance is fixed but the justification is systematically varied across eleven conditions, including neutral, personal experience, causal framing, and worldview-based synthesis. Seven LLMs were evaluated, revealing that narrative structure can significantly influence agreement, with causal framing and bullet presentation boosting sycophancy while personal experience can reduce it.
MindEval2 is a new evaluation framework for AI chatbots in mental health support, comprising Mental Health Quality Rubrics (MHQR) and the MindSim patient simulation harness. MHQR, developed with PhD-level psychologists, provides explicit, actionable criteria for assessing response quality, while MindSim offers realistic, longitudinal patient interactions with a memory component. The study evaluated 18 models, finding that although surface-level quality has improved, guidance calibration remains the main failure across all models.
Large Language Models (LLMs) are increasingly used for interpersonal advice, influencing how users interpret ambiguous social encounters. Existing Theory of Mind evaluations rely on predefined answers, while uncertainty research overlooks LLMs’ preferred explanations and confidence under psychological distress. To address these gaps, the authors introduce MentalLoom, a novel evaluation of LLM mentalisation comprising 604 ambiguous social episodes, matched depression and social anxiety disclosures, and four‑turn dialogues with unchanged event evidence. Across eight LLMs, disclosures initially elicited more negative explanations overall but more positive narrator‑directed interpretations. Scripted repeated questioning showed greater emotional support, stronger positive narrator‑directed endorsement, and higher certainty without new event evidence, demonstrating that user context can reshape interpretive preferences and commitment while the underlying ambiguity remains unresolved.
DialToM is a Theory of Mind benchmark created from naturalistic human-human dialogues, using a multiple-choice format. It introduces a State-Driven Diagnostic Probe that requires models to predict dialogue trajectories based solely on isolated mental-state profiles, without dialogue context. The evaluation shows that large language models are good at inferring mental states (Literal ToM) but struggle to use them for social forecasting (Functional ToM), while a domain expert scores 100% accuracy, highlighting a clear human‑AI gap.
The paper proposes a framework that blends data storytelling with interpretable machine learning (IML) to make AI decisions more understandable to non-experts while protecting sensitive data. It introduces formal concepts such as the DIST Pyramid and the I-P-O Model, and builds an architecture that generates "What-if" and "Why-not" explanations using SHAP values and large language models. A case study on the Boston Housing dataset shows that participants found these data stories significantly more comprehensible and accessible than traditional SHAP visualizations.
arXiv:2607. 18570v1 Announce Type: cross Abstract: Discourse relations provide document structure, critical to language understanding and enabling language model performance and ethicality.