arXiv AI
Sep 15

Hindsight Bias in Clinical Temporal Reasoning: How Future Data Exposure Affects Large Language Model Judgment

The paper introduces a paired benchmark to detect hindsight bias in clinical language models by comparing model responses to questions posed at a clinically relevant cutoff versus the full timeline. It uses 171 case reports (40 sepsis, 131 GLP‑1/diabetes) with both human‑annotated and LLM‑generated time‑series data, evaluating accuracy, hindsight trap rate, answer instability rate, and hindsight bias rate. Results show that exposing models to the full timeline consistently increases hindsight bias, while truncating the timeline mitigates bias without sacrificing accuracy.

By Misaki Matsuura, Sayantan Kumar, Ojas Kadam, Jeremy C. Weiss
arXiv AI
Oct 7

Illusory Pattern Perception Drives Spurious Inference in Large Language Models

The paper "Illusory Pattern Perception Drives Spurious Inference in Large Language Models" reports that large language models (LLMs) exhibit a cognitive bias similar to humans’ tendency to see patterns where none exist. By adapting classic psychological experiments to three tasks, the authors find that LLMs often over‑associate frequent positive attributes with majority groups or large organizations and are prone to constructing causal narratives from ambiguous events. Using a Sparse Autoencoder‑based interpretability framework, the study links these illusory perceptions to holistic frequency perception and analytic cognitive orientation within the models.

By Peihua Mai, Zhuoyan Shao, Xinbao Qiao, Meng Zhang, Xinyue Zhou, Yan Pang
arXiv AI
Aug 28

Can LVLMs Uncover the Truth Behind Visual Illusions? An Analysis of Perceptual and Reasoning Capabilities

The paper introduces Illusion-Reasoning, a benchmark that uses real-world visual illusion images paired with annotated question-answer pairs to evaluate Large Vision Language Models (LVLMs). It argues that current evaluations focus too narrowly on perception or specific domains, and that a joint assessment of perception and reasoning in open-world settings is lacking. Using this benchmark, the authors demonstrate that many LVLMs’ reasoning abilities are less advanced than previously claimed, offering new insights and directions for optimization.

By Liangjie Zhao, Jiaqing Lyu, Kexin Tang, Zecheng Fang, Rong Yin, Yulan Hu, Da Li, Jianing Li