arXiv AI

Interpreting and Steering LLM Agents for Social Simulations

arXiv Computation and Language
Sep 22

SocioVerse2: A Longitudinal Dynamic Social Simulation Framework under a Human-AI Co-evolutionary Paradigm

arXiv:2609.24911v1 Announce Type: new Abstract: Social simulation offers the social sciences an experimental instrument that the real world cannot supply, and generative agents have transformed it by...

By Xinnong Zhang, Jiayu Lin, Jia Wang, Yixu Huang, Xinyi Mou, Yingqian Wu, Jingcong Liang, Shijun Lei, Jianing Shi, Guanying Li, Siyuan Wang, Hanjia Lyu, Zhenfei Yin, Yunlu Yin, Siming Chen, Yulan He, Jiebo Luo, Xuanjing Huang, Liyin Jin, Baohua Zhou, Hanqi Yan, Zhongyu Wei
arXiv AI
Oct 27

How Humans Learn Unfamiliar Worlds: A Think-Aloud Dataset on ARC-AGI-3

The paper presents the first dataset that captures how humans learn and use mechanisms in unfamiliar environments, using ARC‑AGI‑3 as a testbed. It includes 144 trajectories from 64 participants across 25 public games, each paired with time‑synchronized actions and think‑aloud audio, and 250 trajectories from large language model agents with and without harnesses. The dataset is annotated for mechanism triggers, action purposes, errors, and recovery, enabling a comparison of human and LLM cognition, highlighting differences in hypothesis testing, exploration, and use of qualitative representations.

By Shuzhi Liu, Archana Warrier, Dat Nguyen, Chengqi Zheng, Kevin Ellis, Yewen Pu
arXiv AI
Aug 28

Assessing mentalization in humans and large language models

The study evaluates mentalization—the capacity to infer others’ beliefs and intentions—in large language models (LLMs) using two economic games and cognitive computational modeling. Researchers tested 2,099 LLM agents from four model families (DeepSeek, GPT‑4.1, GPT‑5, Gemini 2.0 Flash) against opponents of varying sophistication, comparing their performance to 251 human participants. Results show that LLMs exhibit distinct mentalizing behaviors that vary by model provider and size, with strategic prompting generally enhancing performance; notably, GPT‑5 agents adapt their recursive reasoning depth to match opponent sophistication, outperforming humans in one task.

By Aamir Sohail, Xintong Zhong, Arkady Konovalov, Patricia L. Lockwood, Lei Zhang
arXiv AI
Oct 23

Steering Black-Box Large Language Model Agents with Pitfall Anticipation World Model

The paper introduces the Pitfall Anticipation World Model (PAWM), a selective guidance system for black-box large language model agents. PAWM predicts imminent failures in a bounded multi-step future and intervenes only when a potential pitfall is detected, reducing costly and often uninformative step-by-step predictions. Trained first with supervised fine-tuning on synthetic pitfall-labeled data and then co-trained with a black-box LLM, PAWM boosts Claude Sonnet 4.5’s performance by 24.3% on ScienceWorld and 7.7% on WebArena-Lite, outperforming several baselines.

By Jiongxiao Wang, Han Ding, Youzhi Luo, Yawei Wang, Yun Zhou, Linbo Liu, Panpan Xu
arXiv AI
Jul 3

From Actions to Understanding: Conformal Interpretability of Temporal Concepts in LLM Agents

arXiv:2604. 19775v2 Announce Type: replace Abstract: Large Language Models (LLMs) are increasingly deployed as autonomous agents capable of reasoning, planning, and acting within interactive environments.

By Trilok Padhi, Ramneet Kaur, Krishiv Agarwal, Adam D. Cobb, Daniel Elenius, Manoj Acharya, Colin Samplawski, Alexander M. Berenbeim, Nathaniel D. Bastian, Susmit Jha, Ugur Kursuncu, Anirban Roy