arXiv:2607. 23927v1 Announce Type: new Abstract: A conversational AI that cannot tell its own output from what a user said will treat its own mistakes as user-provided facts.
arXiv:2608. 12387v1 Announce Type: cross Abstract: Positional biases such as recency and primacy effects have been documented in large language models (LLMs), yet the underlying mechanism by which these models make their evaluations remains poorly understood.
The paper introduces a paired benchmark to detect hindsight bias in clinical language models by comparing model responses to questions posed at a clinically relevant cutoff versus the full timeline. It uses 171 case reports (40 sepsis, 131 GLP‑1/diabetes) with both human‑annotated and LLM‑generated time‑series data, evaluating accuracy, hindsight trap rate, answer instability rate, and hindsight bias rate. Results show that exposing models to the full timeline consistently increases hindsight bias, while truncating the timeline mitigates bias without sacrificing accuracy.
The paper "Illusory Pattern Perception Drives Spurious Inference in Large Language Models" reports that large language models (LLMs) exhibit a cognitive bias similar to humans’ tendency to see patterns where none exist. By adapting classic psychological experiments to three tasks, the authors find that LLMs often over‑associate frequent positive attributes with majority groups or large organizations and are prone to constructing causal narratives from ambiguous events. Using a Sparse Autoencoder‑based interpretability framework, the study links these illusory perceptions to holistic frequency perception and analytic cognitive orientation within the models.
arXiv:2608.22232v1 Announce Type: new Abstract: Real-world situation appearances can deviate from their underlying physical states, challenging the reliability of multimodal large language models (ML...
The paper introduces Illusion-Reasoning, a benchmark that uses real-world visual illusion images paired with annotated question-answer pairs to evaluate Large Vision Language Models (LVLMs). It argues that current evaluations focus too narrowly on perception or specific domains, and that a joint assessment of perception and reasoning in open-world settings is lacking. Using this benchmark, the authors demonstrate that many LVLMs’ reasoning abilities are less advanced than previously claimed, offering new insights and directions for optimization.
arXiv:2603.18007v2 Announce Type: replace-cross Abstract: The study explores whether current Large Language Models (LLMs) exhibit Theory of Mind (ToM) capabilities -- specifically, the ability to inf...
arXiv:2508.09852v2 Announce Type: replace-cross Abstract: How can models help people communicate unusual perceptual experiences without changing what they mean? A recognizable image is only part of t...
The paper introduces a benchmark of 83 published works to evaluate whether language models can generate nonlinear narratives—stories where the order of events differs from the order in which they are told. For each work, the authors annotate both chronological and narrative event orders, a causal story graph, character knowledge, and related questions. Experiments show that while models tend to produce more chronological stories than the originals, they only match human performance about half the time when asked to produce a specific nonlinear structure, and readers generally find the default chronological outputs easier to follow.
The paper introduces ToMStories, a dataset of about 6,000 synthetic narrative vignettes designed to test language models’ ability to infer characters’ psychological states such as emotions, fears, motivations, and wounds. Each vignette is paired with either a multiple-choice or open-ended question, and the states are presented implicitly, explicitly, or with conflicting cues. The authors evaluate thirteen open-weight models ranging from 1B to 72B parameters, finding that while the largest models perform on par with humans on multiple-choice tasks, they lag on open-ended questions, revealing simplifications in how models interpret and generate implicit psychological states.
The asymmetry between language production and perception has been well-documented in psycholinguistics. Whether large language models (LLMs) exhibit a functionally analogous distinction remains an open question, particularly given that LLMs rely on the same underlying mechanism (next-token prediction) for both input and output processing.
The paper introduces latent debate, a structured surrogate framework that interprets the internal thinking of Large Language Models (LLMs) on True/False tasks by modeling latent arguments and their interactions. It provides a model- and task-agnostic conceptual framework, instantiated symbolically to approximate LLM decision-making. Empirical results show that latent debate closely matches LLM predictions, offers interpretability, and serves as a strong baseline for hallucination detection, revealing correlations between debate patterns and hallucination risk.