UTILMEM is a new diagnostic benchmark that tests how conversational agents use long‑term memory, focusing on reasoning over dense histories, spotting implicitly relevant memories, synthesizing distributed evidence, and resisting interference from similar distractors. It contains 1,717 instances across five domains and evaluates a range of retrieval‑based and memory‑augmented systems. The study shows that strong performance on traditional factual recall does not guarantee effective memory utilization, highlighting a gap between retrieving information and integrating it into coherent, task‑oriented outputs.
By Peijun Qing, Fobo Shi, Soroush Vosoughi
RealCompanion is a benchmark that evaluates an AI companion’s ability to understand a human over long, real-world conversations. It consists of ten real relationships with 27,218 messages spanning up to 120 days, along with derived files such as a profile, persona, chat ground truth, and a question set that cites the relevant messages. The study finds that past context is rarely needed, memory detection is challenging, and agent systems vary widely in cost while achieving similar persona reconstruction.
By Arman Behnam, Sunglyoung Kim, Liangwei Yang
arXiv:2603.19574v2 Announce Type: replace-cross
Abstract: Conversational AI systems are increasingly used for personal reflection and emotional disclosure, raising concerns about their effects on vul...
By Soorya Ram Shimgekar, Vipin Gunda, Jiwon Kim, Violeta J. Rodriguez, Hari Sundaram, Koustuv Saha
arXiv:2606. 06055v1 Announce Type: new Abstract: Long-term memory enables language model agents to support personalized interactions, but it remains unclear when available memories warrant integration into responses.
By Lingxiang Xu, Jiaoyun Yang, Min Hu, Hongtu Chen, Ning An
arXiv:2607. 14593v1 Announce Type: cross Abstract: As conversational AI systems are designed for repeated use, a central question is how a series of interactions becomes a relationship.
By Ryuichi Sumida, Mao Saeki, Masaki Eguchi, Sadahiro Yoshikawa, Koji Inoue, Tatsuya Kawahara, Yoichi Matsuyama
MIRAGE is a controlled study that examines how multimodal personal agents use historical evidence when conversation state changes. The study keeps evidence, questions, and scoring constant while varying only the conversation state, then checks if agents can determine answerability, recover the correct source, and answer from it. Results across seven multimodal backbones show distinct failure regimes before and after compaction, heavy reliance on context continuity by open-weight models, and mixed effects of retrieval pressure on source attribution.
By Yu Liu, Wenxiao Zhang, Cheng Hu, Cong Cao, Fangfang Yuan, Xinyu Wang, Jin B. Hong, Yanbing Liu
arXiv:2608. 12627v1 Announce Type: cross Abstract: Long-horizon egocentric memory transforms continuous first-person video and audio into a searchable record of past experiences.
By Le Zhang, Ke Sun
arXiv:2609.26780v1 Announce Type: cross
Abstract: Long-term conversational memory in multi-party settings requires more than retrieving relevant content from long-term conversations: it must distingu...
By Haobo Zheng, Tan Tang, Yan Chen, Weijie Wang, Yingcai Wu
arXiv:2602. 01146v2 Announce Type: replace Abstract: Conversational assistants are increasingly integrating long-term memory with large language models (LLMs).
By Sidharth Pulipaka, Oliver Chen, Manas Sharma, Taaha S Bajwa, Vyas Raina, Ivaxi Sheth
arXiv:2602. 24287v2 Announce Type: replace-cross Abstract: In multi-turn conversations, large language models typically condition on the full conversation history: both past user prompts and assistant responses.
By Jenny Y. Huang, Leshem Choshen, Wei Sun, Omar Khattab, Ram\'on Fernandez Astudillo, Mehul Damani, Tamara Broderick, Jacob Andreas
arXiv:2607. 12893v1 Announce Type: new Abstract: Long-term memory has become a foundational capability for LLM-based agents that accompany users across extended, multi-session interactions.
By Xixuan Hao, Zeyu Zhang, Zehao Lin, Yihang Sun, Ziliang Guo, Xichong Zhang, Yuxuan Liang, Feiyu Xiong, Zhiyu Li
Large language model agents are increasingly deployed in human-human interaction settings, such as meeting assistants and clinical documentation systems, where they must observe conversations and retain information for downstream queries. Unlike traditional human-assistant settings, these environments are inherently multimodal, involve complex discourse phenomena such as anaphora and deixis, and contain asynchronous or conflicting information from multiple participants.