arXiv AI

Towards Real-world Human Behavior Simulation: Benchmarking Large Language Models on Long-horizon, Cross-scenario, Heterogeneous Behavior Traces

arXiv AI
Jun 18

How Well Do Large Language Models Capture Human Personality?

arXiv:2606. 18263v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used to simulate human populations via persona prompting, often under the assumptions that richer persona descriptions improve behavioral fidelity, similarly sized attribute combinations are equally simulatable, and persona definitions generalize across tasks.

By Aanisha Bhattacharyya, Yaman Kumar Singla, Rajiv Ratn Shah, Changyou Chen, Jitendra Ajmera
arXiv Computation and Language
Oct 5

CUEing User Simulators: Calibrated User Embeddings for Multi-Turn Benchmarking

The paper introduces Calibrated User Embeddings (CUE), a framework that encodes real user sessions into continuous representations and decodes them into persona commands to steer large language models (LLMs) as user simulators without additional training. CUE enables user-conditioned replay of past interactions and generates novel personas that better align with real-user success rates and failure patterns. Evaluations on the τ²‑Bench dataset show that CUE‑driven simulators commit fewer simulator‑attributed errors, more accurately reproduce real‑user failure modes, and maintain competitive user fidelity across various tasks and LLMs.

By Anjali Kantharuban, Jonas Mueller
arXiv Machine Learning
Oct 22

OmniSimulator: Aligning Small Language Models for Authentic Heterogeneous Behavior Modeling

OmniSimulator is a training framework that enhances small language models for realistic user simulation on short‑video platforms. It structures behavior‑oriented reasoning into five dimensions, selects real user‑behavior data, and uses a teacher model to generate high‑quality reasoning traces, enabling small models to learn behavior‑relevant patterns instead of superficial correlations. The approach yields a 34.33% performance boost for Qwen3‑8B, competitive with Claude‑Sonnet‑4.5, and supports smaller‑scale, self‑hosted deployment.

By Jiawei Chen, Ruoxi Xu, Boxi Cao, Zhuoqun Li, Ruotong Pan, Yunfei Zhang, Zhirui Yang, Jingyan Chen, Tingting Gao, Han Li, Yaojie Lu, Xianpei Han, Le Sun, Xiangyu Wu, Hongyu Lin
arXiv AI
Aug 7

LUNAR: Benchmarking Personalized Large Language Models on UNiversal User BehAvioR Logs

arXiv:2608. 05246v1 Announce Type: new Abstract: Existing personalized LLM benchmarks primarily rely on textual personas or isolated behavioral signals, providing limited evaluation of cross-domain behavioral personalization, where responses must be grounded in heterogeneous daily-life activities.

By Jiahao Zhang, Yongzhi Tong, Zelin Fu, Pengde Zhao, Yanmei Jiang, Jiang Feng, Min Yang
arXiv AI
Aug 28

LiveSim: Simulating Environment-Shaped Users in Multi-Agent Live-Stream Ecosystems

LiveSim is an LLM-based framework designed to simulate users in multi‑agent live‑stream ecosystems. It treats users as editable behavioral hypotheses and refines them through trajectory‑grounded interactions, using discrepancies between simulated and observed trajectories to uncover missing environmental shaping effects. The framework extracts transferable environment‑behavior patterns, stores them in a collective behavioral memory, and demonstrates improved user‑level fidelity and ecosystem‑level analysis on real‑world live‑stream risk‑control data.

By Jiaqi Xu, Yiran Qiao, Jing Chen, Qiwei Zhong, Xiang Ao, Xueqi Cheng
arXiv Machine Learning
Jun 30

Synthetic Interaction Data for Scalable Personalization in Large Language Models

arXiv:2602. 12394v2 Announce Type: replace Abstract: Personalized prompting offers large opportunities for deploying large language models (LLMs) to diverse users, yet existing prompt optimization methods primarily focus on task-level optimization while largely overlooking user-specific preferences and latent constraints of individual users.

By Yuchen Ma, Yue Huang, Wenjie Wang, Xiaonan Luo, Xiangliang Zhang, Stefan Feuerriegel
arXiv Computation and Language
Oct 9

UXBench Pro: Benchmarking Personalized User Experience in Multi-Turn Dialogue Interactions

UXBench Pro is a new benchmark that evaluates personalized user experience in multi-turn dialogue interactions. It contains 1,000 test instances from real user interactions across 12 task scenarios and 82 domains, each paired with a FACTORS user profile describing seven behavioral facets. The benchmark introduces a dual-perspective evaluation: a personalized User Reward Model for third‑person judgment and Sim4Eval, a user simulator for first‑person evaluation across four cognitive state dimensions, along with meta‑benchmarks URMBench and USimBench to assess evaluator reliability.

By Mengze Hong, Zeyang Lei, Wenbo Shang, Xia Zeng, Xiying Zhao, Qi Zhu, Chen Jason Zhang, Di Jiang, Taiming Fu, Qiongyi Zhou, Qinghe Chang, Fubao Zhang, Chenxuan Ma, Minlong Peng, Jinfeng Huang, Zineng Zhou, Jindou Wu, Muge Qi, Sijun He, Xin Cui, Di Liang, Yuan Hua, Davey Chen