The paper introduces the Schema Coherence Inventory (SCI), a 96-item forced-choice test designed to determine whether a belief assigned to a persona-conditioned language model actually organizes its behavior or merely reflects affect. SCI evaluates belief organization across eight levels of Beck's cognitive model, using length‑normalized log‑likelihood scores and a target‑specificity criterion to distinguish true belief influence from mood effects. Validation across five model backbones (1.7B–14B parameters) and nine clinicians shows high reliability and discriminant validity, and a 48‑item short form correlates strongly with the full inventory.
arXiv:2606. 09843v3 Announce Type: replace-cross Abstract: Large language models (LLMs) give stable answers to personality questionnaires, yet these self-reports fail to predict how the models behave.
arXiv:2606. 05970v1 Announce Type: cross Abstract: Large language models are increasingly used for structured extraction from clinical free-text notes, but the sensitivity of their output to upstream configuration choices is less understood than their accuracy on fixed benchmarks.
arXiv:2607. 07670v1 Announce Type: cross Abstract: Large language models hallucinate most about entities they have never seen.
arXiv:2608. 07316v1 Announce Type: cross Abstract: Natural Language Processing (NLP) models predicting mental health outcomes rarely specify what they measure: contextual knowledge, emotional content, or syntactic structure.
The study investigates how different prompt components affect language model responses in psychometric tests. By crossing five distinct baseline personas with five variants of each prompt element—persona wording, task instruction, item wording, and option symbol—the authors measure response shifts using the 1‑Wasserstein distance. Their analysis of 13 small open‑weight language models on the Big Five Inventory and Short Dark Triad reveals that task instruction and option symbol changes often cause more variation than paraphrasing the persona or item, with prompt artifacts explaining over 50% of the variation for many items.
The paper reports a structured case study in which six frontier language models were asked to match handwritten exam answers to the ages of the writers. Three of the models performed at chance on the matching task while still reporting high confidence, and the youngest writer’s answer was misattributed to an adult in up to 90% of trials. Removing the multiple‑choice scaffold and changing institutional framing further degraded accuracy, while a single audience‑awareness instruction improved performance for one model but had no effect on the weakest models.
arXiv:2606. 20205v1 Announce Type: new Abstract: Psychological instruments designed for humans are increasingly used to assign large language models (LLMs) stable psychological profiles that affect their usability, safety assessment, and use as proxies for human participants in research.
arXiv:2606. 07897v1 Announce Type: new Abstract: Current AI models frequently exhibit epistemic sycophancy, endorsing claims to agree with a user.
The paper investigates how much of an AI model’s expressed preferences are due to the model itself versus the instrument (prompt) used to elicit those preferences. By fixing the set of outcomes and models while varying five different prompting instruments across 15 welfare-related outcomes, the study finds that the ranking of outcomes is only moderately generalizable (coefficient 0.348) and that a single instrument’s preference provides little insight into another instrument’s results. The analysis shows that even after removing any single instrument, model, or a subset of outcomes, the overall preference estimate remains robust, yet the variability across instruments remains significant.
arXiv:2606. 09843v1 Announce Type: cross Abstract: Large language models (LLMs) produce stable self-reports on personality inventories, but these self-reports do not predict observed behavior.
arXiv:2607. 13162v1 Announce Type: cross Abstract: What a language model will and will not do is largely set during post-training, but which behaviors it expresses, hides, or resists is not revealed by prompting alone.
arXiv:2607. 02885v1 Announce Type: cross Abstract: Cognitive Behavioral Therapy (CBT) provides a structured framework for understanding a user's mental state by examining the interaction between cognitive and behavioral factors.