← Back to all news
arXiv Machine Learning By Guy Tennenholtz, Ofer Meshi, Amir Globerson, Uri Shalit, Jihwan Jeong, Craig Boutilier

Controllable User Simulation

Read the original on arXiv Machine Learning →

The paper formalizes controllable user simulation as a causal inference problem, showing that standard supervised fine‑tuning on post‑hoc trajectory labels introduces a look‑ahead bias that breaks causal consistency. This bias leads to a phenomenon called controllability collapse, where evaluation metric variance explodes under policy shift. The authors propose theoretical conditions for accurate simulation and practical training mitigations—such as a priori controls, step‑wise dynamic controls, and direct policy‑conditioned learning—to eliminate look‑ahead bias, preserve natural variance, and achieve robust zero‑shot generalization to unseen agent behaviors.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Sep 30

OTROPE: Optimal Transport-based Robust Off-policy Evaluation for Large Language Models

The paper introduces OTROPE, a likelihood‑free method for off‑policy evaluation of large language models (LLMs) that uses optimal transport to align labeled samples from a behavior model with unlabeled samples from a target model in a semantic space. OTROPE corrects human‑labeled residuals with proxy predictors, achieving a doubly robust evaluation without requiring behavior‑policy modeling or density‑ratio estimation. The authors provide theoretical guarantees for consistency and convergence, and demonstrate through synthetic and real LLM tasks that OTROPE outperforms existing baselines and can elevate weaker evaluators to match or exceed stronger ones.

By Liner Xiang, Wenbo Zhang, Hengrui Cai
llms
More like this →
arXiv Computation and Language
Oct 19

Learning User Representations to Improve Simulation Fidelity

The paper introduces UserRepGen-4B, a representation generator that creates natural-language user representations to condition user simulators for more faithful conversation simulation. By formulating a conversation reconstruction task and using hindsight distillation, the system learns to generate representations that improve simulation fidelity across 700 conversations from seven datasets. The improved representations also enhance diversity preservation and user distinction in unseen tasks.

By Shuhaib Mehri, Philippe Laban, Michel Galley, Dilek T\"ur, Jennifer Neville, Jianfeng Gao
llmsefficiency
More like this →
arXiv AI
Jul 17

Step-Level Preference Learning for Generative Agents in Social Simulations

arXiv:2607. 14485v1 Announce Type: new Abstract: Large language model (LLM)-based generative agents simulate human behavior through long-horizon decision-making processes that comprise intermediate steps such as planning, memory retrieval, reflection, and action selection.

By Wenchang Gao, Pingyue Sheng, Lanlan Qiu, Yunfei Ma, Jian Zhao, Baicheng Chen, Kangda Wang, Yuyang Tian, Shunqiang Mao, Tianxing He
llmsagentsfine-tuning
More like this →
arXiv Computation and Language
Oct 8

ToolRACER: A Robust Agentic Conversation Emulation Resource for Agent Training and Evaluation

arXiv:2610.09163v1 Announce Type: new Abstract: Task-oriented conversational agents remain fragile under real world conversation scenarios as they rarely follow a predictable script, especially when...

By Arkajyoti Chakraborty, Aryan Tayal, Ishika Agarwal, Tanner Sorensen, Justin Chiu, Alessandro Di Bari, Neha Gupta, Andreas Stolcke
llmsagentsbenchmarkssafety
More like this →