arXiv:2607. 15883v1 Announce Type: cross Abstract: Large language models are broadly capable, yet in sustained one-to-one conversation they still read as flat: competent, responsive, and somehow not quite the presence of a mind.
By Sebastian Cochinescu
The study examines how users of a major dating platform respond to autonomous LLM agents that converse on their behalf. Using two large surveys, researchers built a latent-variable model showing that willingness to send and receive agent-mediated messages are highly correlated yet distinct. The findings reveal a delegation asymmetry: users are more willing to deploy their own agent than to engage with others’ agents, leading to low overall reciprocity and gender‑directional imbalances in agent interactions.
By Daria Leshchikova, Valentina V. Kuskova, Dmitry Zaytsev, Valerii Klimov
RealCompanion is a benchmark that evaluates an AI companion’s ability to understand a human over long, real-world conversations. It consists of ten real relationships with 27,218 messages spanning up to 120 days, along with derived files such as a profile, persona, chat ground truth, and a question set that cites the relevant messages. The study finds that past context is rarely needed, memory detection is challenging, and agent systems vary widely in cost while achieving similar persona reconstruction.
By Arman Behnam, Sunglyoung Kim, Liangwei Yang
arXiv:2609.26035v1 Announce Type: new
Abstract: Conversational agents often express answers in a uniformly confident register. We test whether expressed uncertainty, provenance-aware assertion, and e...
By Sebastian Cochinescu
GAUGE is a new offline protocol that evaluates whether the common practice of using an LLM-as-a-judge to rank task‑oriented agents actually aligns with a verifiable reward. Across 25 agents from six providers on two benchmarks, GAUGE finds that user satisfaction scores are largely uncorrelated with task success, and that the judge’s ranking loses precision when agents are closely matched in performance. The study highlights a gap between ranking validity and construct validity in current evaluation practices.
By Umesh Bodhwani, Thanh Tran, Kai Wei
The paper introduces HICO-Bench, a benchmark of 180 everyday coordination scenarios where two agents with private information converse and then record a shared plan. A program verifies that the plans are feasible and an LLM judge checks that the conversation supports the plan, defining a Verified Success. Using Paired-Credit Policy Optimization (PaCPO), the authors improve the Verified Success rate of Qwen3-8B from 20.4% to 28.3%.
By Caixin Kang, Ke Wang, Yuchuan Wu, Xiaoqiang Wang, Ling Li, Bingxue Qiu, Yongbin Li, Yoichi Sato