arXiv AI
Jun 17
Using Cognitive Models to Improve Language Model Simulation of Human Persuasion Games
arXiv:2606. 17657v1 Announce Type: new Abstract: People make decisions differently in strategic interactions.
Assessing mentalization in humans and large language models
The study evaluates mentalization—the capacity to infer others’ beliefs and intentions—in large language models (LLMs) using two economic games and cognitive computational modeling. Researchers tested 2,099 LLM agents from four model families (DeepSeek, GPT‑4.1, GPT‑5, Gemini 2.0 Flash) against opponents of varying sophistication, comparing their performance to 251 human participants. Results show that LLMs exhibit distinct mentalizing behaviors that vary by model provider and size, with strategic prompting generally enhancing performance; notably, GPT‑5 agents adapt their recursive reasoning depth to match opponent sophistication, outperforming humans in one task.
Related stories
arXiv AI
Aug 11
Avalon-ToM-Bench: Evaluating Fine-Grained Theory of Mind via Asymmetric Game Mechanics
arXiv:2608. 09638v1 Announce Type: new Abstract: Theory of Mind (ToM) is essential for agent interactions, yet existing evaluations either rely on static scenarios that oversimplify mental-state reasoning or interactive settings that provide limited diagnostic insight.
arXiv AI
Jul 2
Large language models replicate and predict human cooperation across experiments in game theory
arXiv:2511. 04500v3 Announce Type: replace Abstract: Large language models (LLMs) are increasingly deployed as decision-making agents in high-stakes domains and as imitators of human behavior in the social and behavioral sciences.
arXiv AI
Sep 17
What Counts as Strategic Reasoning? A Systematic Mapping of Chess Research on Humans, Engines, and Language Models
The paper presents a systematic mapping of recent chess research involving humans, engines, neural and reinforcement‑learning systems, large language models (LLMs), and hybrid approaches. It identifies 84 core study families and classifies them by agent type, strategic‑reasoning stages, and evaluation dimensions, highlighting a strong focus on situation assessment, evaluation, and action selection while noting gaps in planning, explanation, metacognition, and human–AI collaboration. The study also distinguishes hybrid systems by integration timing and cautions that improved human performance in evaluations does not automatically imply human–AI synergy.
arXiv AI
Aug 18
The Fragility of Strategic Thinking in Large Language Models
arXiv:2510. 10813v2 Announce Type: replace Abstract: Large Language Models (LLMs) are increasingly applied to domains that require reasoning about other agents' behavior, such as negotiation, policy design, and market simulation.
arXiv AI
Jul 14
People use fast and flat simulation to reason about new games
arXiv:2510. 11503v2 Announce Type: replace-cross Abstract: Games have long been a microcosm for studying planning and reasoning in both natural and artificial intelligence (AI), often focusing on expert-level or even super-human play.
Hugging Face Trending Papers
Jun 30
Theory of Mind and Persuasion Beyond Conversation: Assessing the Capacity of LLMs to Induce Belief States via Planning and Action
Theory of Mind (ToM) benchmarks for Large Language Models (LLMs) typically rely on passive question-answering formats, but the deployment of LLMs in increasingly agentic and autonomous forms demands new evaluations. In this paper we evaluate an agent's ability to induce specific belief states in other agents by taking actions rather than using conversational persuasion, a capability we call Non-Conversational Planning ToM (NCP-ToM).
arXiv AI
5d ago
Beyond Successful Play: Diagnosing Opponent Modeling in Large Language Models
arXiv:2610.17310v1 Announce Type: new Abstract: Language-model agents are expected to adapt to others whose behavior changes through interaction. Yet adaptation may rely on learned behavioral pattern...
arXiv AI
1d ago
Language Model Goal Selection Differs from Humans' in a Self-Directed Learning Task
The study examines whether large language models (LLMs) can accurately emulate human goal selection in a self‑directed learning task. Five models—GPT‑5, Gemini 2.5 Pro, Claude Sonnet 4.5, Qwen3 32B, and Centaur—were tested and found to diverge significantly from human behavior, often exploiting a single solution or performing poorly, with little variation across instances. Techniques such as chain‑of‑thought reasoning and persona steering offered only limited improvements, and the results were consistent across different experimental settings.
arXiv AI
Aug 25
Effects of Theory of Mind and Prosocial Beliefs on Steering Human-Aligned Behaviors of LLMs in Ultimatum Games
The study examines how Theory of Mind (ToM) reasoning and prosocial beliefs influence large language models (LLMs) in the ultimatum game. By initializing LLM agents with Greedy, Fair, or Selfless beliefs and applying chain‑of‑thought or varying levels of ToM reasoning, the authors ran 2,700 simulations across several models, including o3‑mini and DeepSeek‑R1 Distilled Qwen 32B. Results show that ToM‑enhanced LLMs align more closely with human decision patterns, exhibit greater consistency, and achieve better negotiation outcomes, with Llama 3.3 70B producing the most belief‑consistent reasoning. whyItMatters":"The findings clarify the importance of incorporating Theory of Mind into LLMs to improve their alignment with human norms in cooperative decision‑making tasks."
arXiv AI
Jun 6
CLASH: Evaluating Language Models on Judging High-Stakes Dilemmas from Multiple Perspectives
arXiv:2504. 10823v4 Announce Type: replace-cross Abstract: Navigating dilemmas involving conflicting values is challenging even for humans in high-stakes domains, let alone for AI, yet prior work has been limited to everyday scenarios.
arXiv AI
Aug 14
LLMs Are Not Good Strategists, Yet Memory-Enhanced Agency Boosts Reasoning
arXiv:2608. 12626v1 Announce Type: cross Abstract: Strategic reasoning in Large Language Models (LLMs) within long-horizon environments is often limited by inconsistent subgoals.