Disclosure-Gated User Simulation for Companion-Agent Evaluation

The paper introduces a disclosure‑gated user simulator for companion‑agent evaluation, where information release is conditioned on the agent’s behavior through a five‑level gate system. The authors train the simulator on synthetic and real data, audit its performance, and demonstrate that it preserves ranking order and score stability across 12 tested systems. Their released simulator achieves a 0.993 correlation with the original benchmark’s simulator and outperforms prompting a frontier model, which only shifts scores upward without affecting rankings.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• llms
• agents
• benchmarks