arXiv AI By Sion Weatherhead, Ben Newell, Aaron Belbasis, Flora Salim

Imperfectly Imperfect: Evaluating Graded Cognitive Diversity in LLM Persona Simulations

Read the original on arXiv AI →

The paper evaluates whether large language model (LLM) agents can exhibit graded cognitive diversity when simulating human-like behaviour. Using a cognition-suite that includes new Cognitive Reflection Test (CRT) and Behavioural Reflection Test (BRT) alongside established measures, the study finds that while agents can replicate trait distributions, they fail to mirror human variation in task performance and downstream behaviour. Claude Sonnet 5 performs best but still shows significant discrepancies compared to human data.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computation and Language
Oct 23

On the Behavioral Traits of LLM Agents

The paper introduces A-B-D, a method that infers personality traits of large language model agents from their behavioral data, combining functional actions and linguistic outputs across 345,667 real-world trajectories of 80 models. After filtering for stability and discriminability, 79 features are reduced to six reliable factors—two functional and four linguistic—revealing differences such as Claude‑Sonnet‑4.6’s planfulness versus GPT‑5.5’s low energy. The study also demonstrates a weak correlation between these inferred traits and agents’ self-reported Big Five scores, highlighting a knowledge‑action gap.

By Haokai Zhao, Jie Gao, Yunze Xiao, Xintao Wang, Weihao Xuan, Aditya Joshi, Mark Dredze, Jen-tse Huang
arXiv Computation and Language
Oct 19

The Chameleon's Limit: Investigating Persona Collapse and Homogenization in Large Language Models

The paper introduces the concept of Persona Collapse, where large language models (LLMs) produce clustered, non-diverse responses in multi-agent simulations. It defines three diagnostic metrics—Coverage, Uniformity, and Complexity—to quantify this phenomenon across eight LLMs on tasks such as personality, moral reasoning, and self-introduction. Findings show that persona collapse is universal but varies by task, cannot be alleviated by sampling more personas, and that higher per-persona fidelity leads to more polarized populations, while persona-targeted fine-tuning further reduces diversity.

By Yunze Xiao, Vivienne J. Zhang, Chenghao Yang, Weihao Xuan, Jen-tse Huang