arXiv AI

Local Predictability and Collective Fidelity in LLM-Agent Societies

The paper investigates whether compact surrogate models can reduce the cost of simulating large language model (LLM) societies while still reproducing their collective behavior. Using 9,455 published trajectories and new opinion‑dynamics experiments, it finds that incorporating neighbor information improves individual predictions across all 16 public‑data settings and enhances pooled collective forecasts on held‑out questions, though the collective gains vary with transfer conditions. Additional tests on 24 new statements do not confirm earlier contrasting history effects, and Qwen shows benefit from history only after three observed rounds, underscoring the need for direct collective validation, explicit limits on available observations, and comparisons with simple baselines.

arXiv Machine Learning
Jul 7

Social Networks of LLM Agents

arXiv:2607. 03695v1 Announce Type: new Abstract: Large language model (LLM) agents are increasingly deployed in interacting populations, raising the question of what such populations come to believe collectively.

By Kaixuan Liu, Guojun Xiong, Weinan Zhang, Shengpu Tang
arXiv Computation and Language
Sep 11

The PIMMUR Principles: Ensuring Validity in Collective Behavior of LLM Societies

The study audits 576 LLM-based social simulations from 350 papers using the PIMMUR framework, which evaluates agent profile, interaction, memory, minimal control, unawareness, and realism. Results show that PIMMUR principles are met more often than minimal control, unawareness, and realism, with frontier LLMs correctly identifying the underlying experiment in 65.2% of cases and half of prompts pre‑determining outcomes. Reproducing five experiments revealed that many reported collective phenomena disappear or reverse when PIMMUR principles are enforced, suggesting that apparent emergent behaviors may be methodological artifacts rather than genuine social dynamics.

By Jiaxu Zhou, Jen-tse Huang, Xuhui Zhou, Man Ho Lam, Xintao Wang, Hao Zhu, Wenxuan Wang, Maarten Sap
arXiv Machine Learning
Sep 18

Digital Twins for Opinion Dynamics: A Generative LLM Framework for Social Networks

The paper introduces a digital‑twin framework that simulates opinion dynamics in real Twitter networks by assigning agents attributes such as persona, emotions, centrality, stubbornness, and influence, and using Mistral‑7B to update opinions based on memory and social exposure. Validation on COVID‑19 and U.S. election 2020 datasets shows the framework reproduces opinion trajectories, reducing prediction error by over 50% compared to classical baselines, and improves structural alignment and polarization dynamics. Ablation studies reveal that agent attributes, memory, and social exposure all contribute to predictive fidelity, with agent attributes being the most critical.

By Omran Berjawi, Giuseppe Fenza, Rida Khatoun, Sherali Zeadally
arXiv AI
Jun 30

Diversity is the Strength of the AI Crowd

arXiv:2606. 29661v1 Announce Type: new Abstract: Top AI forecasting systems are approaching superforecaster-level accuracy on future world events, but still rely primarily on off-the-shelf LLMs combined with forecasting-specific context gathering and scaffolding.

By Matthew Aitchison, Scott Jeen, Toby Shevlane, Ben Day