arXiv AI

Imperfectly Imperfect: Evaluating Graded Cognitive Diversity in LLM Persona Simulations

The paper evaluates whether large language model (LLM) agents can exhibit graded cognitive diversity when simulating human-like behaviour. Using a cognition-suite that includes new Cognitive Reflection Test (CRT) and Behavioural Reflection Test (BRT) alongside established measures, the study finds that while agents can replicate trait distributions, they fail to mirror human variation in task performance and downstream behaviour. Claude Sonnet 5 performs best but still shows significant discrepancies compared to human data.

arXiv Computation and Language
Oct 23

On the Behavioral Traits of LLM Agents

The paper introduces A-B-D, a method that infers personality traits of large language model agents from their behavioral data, combining functional actions and linguistic outputs across 345,667 real-world trajectories of 80 models. After filtering for stability and discriminability, 79 features are reduced to six reliable factors—two functional and four linguistic—revealing differences such as Claude‑Sonnet‑4.6’s planfulness versus GPT‑5.5’s low energy. The study also demonstrates a weak correlation between these inferred traits and agents’ self-reported Big Five scores, highlighting a knowledge‑action gap.

By Haokai Zhao, Jie Gao, Yunze Xiao, Xintao Wang, Weihao Xuan, Aditya Joshi, Mark Dredze, Jen-tse Huang
arXiv Computation and Language
Oct 19

The Chameleon's Limit: Investigating Persona Collapse and Homogenization in Large Language Models

The paper introduces the concept of Persona Collapse, where large language models (LLMs) produce clustered, non-diverse responses in multi-agent simulations. It defines three diagnostic metrics—Coverage, Uniformity, and Complexity—to quantify this phenomenon across eight LLMs on tasks such as personality, moral reasoning, and self-introduction. Findings show that persona collapse is universal but varies by task, cannot be alleviated by sampling more personas, and that higher per-persona fidelity leads to more polarized populations, while persona-targeted fine-tuning further reduces diversity.

By Yunze Xiao, Vivienne J. Zhang, Chenghao Yang, Weihao Xuan, Jen-tse Huang
arXiv AI
Sep 30

Beyond Symmetric Agents: Cognitive Diversity and Multi-Agent Debate in Small Language Models

The study evaluates multi‑agent debate (MAD) in small language models, testing whether cognitive diversity—via personas, sampling temperature, or model identity—drives performance gains. Across 23 models, five tasks, and over 5,500 runs, MAD consistently outperforms single‑model inference but, when matched for generation budget, it ties or falls behind self‑consistency sampling, with persona prompting actually reducing accuracy. The authors find that MAD’s benefits largely stem from the first answer exchange and that many reported gains are due to ensemble‑sampling effects rather than true diversity, highlighting the need for budget‑matched, contamination‑checked baselines. whyItMatters":"The findings clarify that MAD’s perceived advantages may be overestimated and that future debate mechanisms must be evaluated against rigorous, budget‑matched baselines to ensure genuine performance improvements."

By Leonardo Ferreira, Gardenia Liu, Kaden Zheng
arXiv AI
Oct 19

Language Model Goal Selection Differs from Humans' in a Self-Directed Learning Task

The study examines whether large language models (LLMs) can accurately emulate human goal selection in a self‑directed learning task. Five models—GPT‑5, Gemini 2.5 Pro, Claude Sonnet 4.5, Qwen3 32B, and Centaur—were tested and found to diverge significantly from human behavior, often exploiting a single solution or performing poorly, with little variation across instances. Techniques such as chain‑of‑thought reasoning and persona steering offered only limited improvements, and the results were consistent across different experimental settings.

By Gaia Molinaro, Dave August, Danielle Perszyk, Anne G. E. Collins