arXiv Computation and Language September 11, 2026 By Jiaxu Zhou, Jen-tse Huang, Xuhui Zhou, Man Ho Lam, Xintao Wang, Hao Zhu, Wenxuan Wang, Maarten Sap

The PIMMUR Principles: Ensuring Validity in Collective Behavior of LLM Societies

Read the original on arXiv Computation and Language →

The study audits 576 LLM-based social simulations from 350 papers using the PIMMUR framework, which evaluates agent profile, interaction, memory, minimal control, unawareness, and realism. Results show that PIMMUR principles are met more often than minimal control, unawareness, and realism, with frontier LLMs correctly identifying the underlying experiment in 65.2% of cases and half of prompts pre‑determining outcomes. Reproducing five experiments revealed that many reported collective phenomena disappear or reverse when PIMMUR principles are enforced, suggesting that apparent emergent behaviors may be methodological artifacts rather than genuine social dynamics.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv AI

3d ago

From the Fluency Fallacy to the Micro-to-Macro Validity Gap: Opportunities and Pitfalls of LLMs in Social Simulation

arXiv:2507.19364v3 Announce Type: replace Abstract: The integration of Large Language Models (LLMs) into social simulation has generated considerable enthusiasm, but also raises substantial methodolo...

By Patrick Taillandier, Jean Daniel Zucker, Arnaud Grignard, Benoit Gaudou, Nghi Quang Huynh, Haojia Kong, Alexis Drogoul

llms agents safety

arXiv AI

Jul 2

Large language models replicate and predict human cooperation across experiments in game theory

arXiv:2511. 04500v3 Announce Type: replace Abstract: Large language models (LLMs) are increasingly deployed as decision-making agents in high-stakes domains and as imitators of human behavior in the social and behavioral sciences.

By Andrea Cera Palatsi, Samuel Martin-Gutierrez, Ana S. Cardenal, Max Pellert

llms agents safety

arXiv Computation and Language

Aug 31

Benchmarking large language model agent societies against human behavioural distributions

The paper introduces SILICA, an open instrument designed to evaluate whether large language model (LLM) agent societies replicate human behavioural distributions. Using five environments with human‑anchored data and perturbations, the study finds that most LLMs only match human behaviour at initial stages, failing to reproduce end‑state cooperation or correct acceptance thresholds. The results suggest that current LLM societies can support exploratory claims but do not yet reliably emulate human social dynamics.

By Raad Bin Tareaf

llms agents benchmarks

arXiv AI

Jun 30

Collective cooperation without individual fidelity in LLM agents

arXiv:2606. 30454v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as agents in simulations of social systems, yet it remains unclear when their behavior can be interpreted as a faithful proxy for human decision-making.

By Henrique Ferraz de Arruda, Carlos Gracia L\'azaro, Alberto Aleta, Yamir Moreno

llms agents benchmarks

arXiv Computation and Language

Sep 1

Do LLMs Change Their Minds Like Humans? Diagnosing Human--LLM Divergence in Single-Turn Persuasion Judgments

arXiv:2608.29803v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly deployed as proxies for human participants in social simulations, yet whether they update their beliefs...

By Lin Chen, Yitong Chen, Yong Li

llms

arXiv AI

3d ago

The Emergence of Social Science of Large Language Models

arXiv:2509.24877v3 Announce Type: replace Abstract: The social science of large language models (LLMs) examines how these systems evoke mind attributions, interact with one another, and transform hum...

By Xiao Jia, Zhanzhan Zhao

llms rag agents safety

One email a morning, machine-written

Related stories

From the Fluency Fallacy to the Micro-to-Macro Validity Gap: Opportunities and Pitfalls of LLMs in Social Simulation

Large language models replicate and predict human cooperation across experiments in game theory

Benchmarking large language model agent societies against human behavioural distributions

Collective cooperation without individual fidelity in LLM agents

Do LLMs Change Their Minds Like Humans? Diagnosing Human--LLM Divergence in Single-Turn Persuasion Judgments

The Emergence of Social Science of Large Language Models