The paper introduces user simulators that provide intrinsic feedback by incorporating latent evaluation states such as internal thoughts, reactions, and scalar ratings. These simulators, trained on real user–assistant interaction logs, outperform prompted baselines in realism and critiquing behaviors, and their turn‑level evaluations better align with human judgments. Additionally, the intrinsic feedback serves as an effective reward signal for multi‑turn reinforcement learning, reducing compute requirements by over 50% compared to setups using external judge LLMs.
By Tarek Naous, Craig Boutilier, Wei Xu, Ofer Meshi
arXiv:2609.15972v1 Announce Type: cross
Abstract: As language models become more capable, long-term collaboration in learning, reasoning, and decision-making calls for a deeper understanding of the p...
By Zixuan Wang, Yufan Zhou, Jinzhou Tang, Xinle Yu, Chengjun Wu, Lyumanshan Ye, Zhaoxiang Feng, Letian Peng, Adyasha Patra, Fan Bai, Enze Ma, Zhengding Hu, Jianyang Gu, Zhao Wang, Yufei Ding, Jingbo Shang, Tianmin Shu, Zhiting Hu, Zhen Wang
arXiv:2605.12894v2 Announce Type: replace-cross
Abstract: Large Language Model (LLM) agents are increasingly deployed in settings where they interact with diverse users, including those who are uncle...
By Harshita Chopra, Kshitish Ghate, Aylin Caliskan, Tadayoshi Kohno, Chirag Shah, Natasha Jaques
arXiv:2608. 12253v1 Announce Type: cross Abstract: Multi-agent reinforcement learning for human-AI interaction typically relies on a single large language model to simulate user behavior.
By Simon Yu, Nicholas Tomlin, Marwa Abdulhai, Ximing Lu, Derek Chong, Abe Hou, Dilara Soylu, Sergey Levine, Christopher D. Manning, Weiyan Shi
arXiv:2606. 14199v1 Announce Type: cross Abstract: Large language models are increasingly deployed as human simulators for interactive evaluation and social simulation.
By Xuhui Zhou, Weiwei Sun, Weihua Du, Jiarui Liu, Haojia Sun, Qianou Ma, Tongshuang Wu, Yiming Yang, Maarten Sap
The paper introduces Calibrated User Embeddings (CUE), a framework that encodes real user sessions into continuous representations and decodes them into persona commands to steer large language models (LLMs) as user simulators without additional training. CUE enables user-conditioned replay of past interactions and generates novel personas that better align with real-user success rates and failure patterns. Evaluations on the τ²‑Bench dataset show that CUE‑driven simulators commit fewer simulator‑attributed errors, more accurately reproduce real‑user failure modes, and maintain competitive user fidelity across various tasks and LLMs.
By Anjali Kantharuban, Jonas Mueller