arXiv Computation and Language By Zanwu Liu, Dan Zhang, Xiaodi Wang, Yongbo Mao, Xiaoyang Guo, Yonglin Wu, Peiyue Jiang, Yangyang Wei, Zecheng Lin

Stop Training Against Soft Users: Mentalized Adversarial Self-Play for Proactive Dialogue

Read the original on arXiv Computation and Language →

The paper introduces Mentalized Adversarial Self-Play (MASP), a framework for training proactive dialogue agents by modeling the user as a rationally resisting participant with an evolving mental state. MASP separates teacher and student mentalization, alternates self‑play updates for system and user policies, and treats robustness to rational resistance as a training objective. Experiments on proactive dialogue benchmarks show MASP improves success rate, rationality‑constrained success, reduces average turns, and performs better against hard users compared to SFT and Planner‑only approaches.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv AI
Jun 4

Unlocking Proactivity in Task-Oriented Dialogue

arXiv:2605. 22240v2 Announce Type: replace Abstract: Proactive task-oriented dialogue (TOD), such as outbound sales, demands a persuasive agent that actively probes the user's concerns and steers the conversation toward acceptance within a bounded number of turns.

By Azure Zhang, Ning Gao, Yuqin Dai, Ruiyuan Wu, Jinpeng Wang, Rena Wei Gao, Bingdong Tan, Shuzheng Gao, Zongjie Li, Chaozheng Wang
arXiv AI
Oct 9

MIMESIS: Learning User Simulators as Training Environments for Interactive Agents

MIMESIS is a user simulator designed to train interactive language agents by mimicking real user behavior. It is trained on human conversations with explicit reasoning supervision and incorporates 13 realistic behavioral patterns. Experiments show that agents trained with MIMESIS outperform those trained with GPT-5.5 across multiple unseen simulators, and the proposed Coached On-Policy Self-Distillation further improves agent performance by providing dense, token-level supervision.

By Hoang Phan, Dat Huynh, Andrey Zhmoginov, Qi Zeng, Wancen Mu, Yue Cao, Shengjie Bi, Yun He, Changdae Oh, Deren Lei
arXiv AI
Jun 30

Pushing Forward Pareto Frontiers of Proactive Agents with Behavioral Agentic Optimization

arXiv:2602. 11351v2 Announce Type: replace Abstract: Proactive large language model (LLM) agents aim to actively plan, query, and interact over multiple turns, enabling efficient task completion beyond passive instruction following and making them essential for real-world, user-centric applications.

By Yihang Yao, Zhepeng Cen, Haohong Lin, Shiqi Liu, Zuxin Liu, Jiacheng Zhu, Zhang-Wei Hong, Laixi Shi, Ding Zhao
arXiv Computation and Language
Oct 12

Training User Simulators with Latent Intrinsic Feedback

The paper introduces user simulators that provide intrinsic feedback by incorporating latent evaluation states such as internal thoughts, reactions, and scalar ratings. These simulators, trained on real user–assistant interaction logs, outperform prompted baselines in realism and critiquing behaviors, and their turn‑level evaluations better align with human judgments. Additionally, the intrinsic feedback serves as an effective reward signal for multi‑turn reinforcement learning, reducing compute requirements by over 50% compared to setups using external judge LLMs.

By Tarek Naous, Craig Boutilier, Wei Xu, Ofer Meshi