arXiv AI

The Internal Anatomy of Strategic Choice in Large Language Models

arXiv Computation and Language
Aug 31

Benchmarking large language model agent societies against human behavioural distributions

The paper introduces SILICA, an open instrument designed to evaluate whether large language model (LLM) agent societies replicate human behavioural distributions. Using five environments with human‑anchored data and perturbations, the study finds that most LLMs only match human behaviour at initial stages, failing to reproduce end‑state cooperation or correct acceptance thresholds. The results suggest that current LLM societies can support exploratory claims but do not yet reliably emulate human social dynamics.

By Raad Bin Tareaf
arXiv Machine Learning
Nov 2

GENSTRAT: Toward a Science of Strategic Reasoning in Large Language Models

GENSTRAT is a system that evaluates strategic reasoning in large language models (LLMs) by procedurally generating two-player zero-sum imperfect-information betting games on demand. Each game is scored across five complexity axes—state space, temporal contingency, opponent dependence, private information, and precision—and assessed via Monte Carlo simulation. Using this generator, the authors created a 50-game benchmark, ran a tournament of twelve frontier and open-weight models with over 40,000 LLM vs. LLM matches, and found that the performance gap on the hardest games is 1.7 times larger than across all games, with higher thinking budgets, opponent strategy disclosure, and tailored reasoning improving outcomes.

By Vartan Shadarevian, Kia Ghods, Alex Kenich, Anany Kotawala
arXiv AI
Oct 27

Jointly Modeling Preferences and Imperfect Decision-Making in Large Language Models

The paper introduces the Preference-Behavior Model (PBM), which jointly infers Large Language Model (LLM) preferences and imperfect decision-making mechanisms from observed choices. By applying PBM to three social dilemma games—Public Goods, Collective-Risk, and Common-Pool Resource—the authors show that preferences derived from one dilemma can predict behavior in others, whereas traditional behavioral economic elicitation instruments fail to do so consistently. The study also evaluates how well inferred preferences align with persona instructions, revealing both broad agreement and specific divergences across dimensions, LLMs, and games.

By Axel Abels, Elias Fern\'andez Domingos, Tom Lenaerts
arXiv AI
Aug 20

How AI Prompts Can Teach Us About the Structure of Human Behavior

The paper presents an AI-based method that uses a large language model to emulate human decision-making by assigning it a "type vector" describing traits such as Altruism and Risk Aversion. By varying these dimensions and values, the authors fit the model to over 119,000 decisions from 78,657 participants in 10 classic economic games, finding that three dimensions—Risk Aversion, Strategic Sophistication, and Trust—sufficiently capture human behavior. The resulting type clusters, fewer than a dozen, predict behavior in new games, suggesting a low-dimensional, portable representation of human behavior across diverse settings.

By Matthew O. Jackson, Benjamin S. Manning, Yutong Xie, Walter Yuan, Qiaozhu Mei
arXiv AI
Jun 9

Payoff scaling shapes cooperation in LLM agents across languages

arXiv:2601. 19082v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly deployed as autonomous agents that negotiate, coordinate, and act on behalf of users.

By Trung-Kiet Huynh, Dao-Sy Duy-Minh, Thanh-Bang Cao, Phong-Hao Le, Hong-Dan Nguyen, Phu-Quy Nguyen-Lam, Minh-Luan Nguyen-Vo, Hong-Phat Pham, Phu-Hoa Pham, Thien-Kim Than, Chi-Nguyen Tran, Huy Tran, Gia-Thoai Tran-Le, Alessio Buscemi, Le Hong Trang, The Anh Han