UTILMEM is a new diagnostic benchmark that tests how conversational agents use long‑term memory, focusing on reasoning over dense histories, spotting implicitly relevant memories, synthesizing distributed evidence, and resisting interference from similar distractors. It contains 1,717 instances across five domains and evaluates a range of retrieval‑based and memory‑augmented systems. The study shows that strong performance on traditional factual recall does not guarantee effective memory utilization, highlighting a gap between retrieving information and integrating it into coherent, task‑oriented outputs.
By Peijun Qing, Fobo Shi, Soroush Vosoughi
arXiv:2606. 04442v1 Announce Type: cross Abstract: AI systems increasingly need to combine two demanding capabilities: navigating multi-session conversation history and performing deep reading comprehension within long documents.
By Qiyang Xie, Jialun Wu, Xinjie He, Su Liu, Shuai Xiao, Zhiyuan Lin, Weikai Zhou
JustMem is a new memory system for long‑term conversational AI that stores conversation history as compact atomic memories and adapts its access strategy to each query. It introduces two dimensions of memory access—discovery breadth and reading fidelity—implemented through LOOKUP for local evidence, COMPOSE for distributed evidence, and REPLAY for fidelity‑sensitive evidence. Experiments on LoCoMo and LongMemEval‑S show that JustMem outperforms competing memory systems in accuracy and recall while using fewer generative‑model tokens for memory construction and inference.
By Guanhua Chen, Yanting Wang, Wenjing Zhi, Lei Sha
The paper introduces SCALE-QA, a new QA benchmark that tests conversational memory in flat, unsegmented multi‑topic threads by requiring agents to infer which earlier episode supports a later task decision. The dataset contains 3,000 audited questions across ten domains, uses deterministic four‑way multiple‑choice grading, and includes a runtime builder for reproducibility. The authors also propose Temporal‑Semantic Interleaved Memory Reconstruction (TSIM), a hierarchical memory stack that segments turns into coherent episodes and indexes them with deterministic summaries and cluster‑routing views, achieving significant accuracy gains over strong RAG baselines and long‑context LLMs.
By Zhexi Feng, Ruiyi Zhang, Yongbo Yang, Pengtao Xie
AI systems increasingly need to combine two demanding capabilities: navigating multi-session conversation history and performing deep reading comprehension within long documents. Yet no existing benchmark evaluates both simultaneously.
arXiv:2608. 03463v1 Announce Type: new Abstract: Long-term memory is essential for LLM-based agents to sustain interactions and reliably leverage distant history.
By Yuxin Liao, Le Wu, Min Hou, Hao Liu, Han Wu, Zishu Wang
arXiv:2609.07093v2 Announce Type: replace
Abstract: Retrieval-augmented generation (RAG) enables large language models (LLMs) to answer questions by accessing external knowledge and has been widely a...
By Yifan Wang, Xinkui Lin, Yongxiu Xu, Shen Gao, Ruochen Yang, Kun Huang, Yubin Wang, Jie Wu, Wei Liu, Jian Luan, Hongbo Xu, Shuo Shang
arXiv:2607. 12893v1 Announce Type: new Abstract: Long-term memory has become a foundational capability for LLM-based agents that accompany users across extended, multi-session interactions.
By Xixuan Hao, Zeyu Zhang, Zehao Lin, Yihang Sun, Ziliang Guo, Xichong Zhang, Yuxuan Liang, Feiyu Xiong, Zhiyu Li
arXiv:2609.22231v1 Announce Type: new
Abstract: Long-horizon interactions with LLM-based assistants require memory systems that preserve and update user states, preferences, and interaction histories...
By Zeyu Liu, Jian Zhong, Rongduo Han, Ziyang Wu, Shunye Tang, Chenghao He, Yaxuan Yang, Yihang Qiu, Ailing Wang, Xiao Liang, Guohuan Xie, Xiaokang Xue, Gongchen Li, Haining Zhang, Wei Wang
arXiv:2605. 28831v2 Announce Type: replace-cross Abstract: Long-horizon memory question answering often requires sparse evidence from heterogeneous histories, including events, object states, visual observations, temporal relations, and causal steps.
By Encheng Su, Jianyu Wu, Jinouwen Zhang, Qiucheng Yu, Chen Tang, Pengze Li, Lintao Wang, Aoran Wang, Xinzhu Ma, Shixiang Tang, Yizhou Wang, Houqiang Li
The paper introduces LOCOMO-CONV, a conversational memory benchmark that expands on the existing LoCoMo dataset with four query styles—dialog, implicit, counterfactual, and composed—designed to evaluate memory systems in realistic conversational settings. Experiments across five memory systems reveal that conversational framing uncovers significant retrieval gaps missed by traditional QA benchmarks, particularly for implicit and composed queries, and that strong retrieval does not necessarily translate into higher response quality. The study also highlights silent grounding in implicit queries, where memory enhances contextual grounding without explicitly presenting the gold fact, suggesting a need for reasoning-based memory elaboration.
By Wen-Yu Chang, Yun-Nung Chen
Entity-Memory graph retrieval preserves dialogue turns as verbatim memory nodes, links repeated mentions via shared entities, and connects adjacent memories with chronological edges. During retrieval, the system gates through entities, fuses semantics, and performs one‑hop chronological recovery before dense backfill, allowing it to keep neighboring memories that dense cosine ranking might miss. On 1,986 questions from ten LoCoMo conversations, this graph retrieval method increases official evidence recall at top‑k 25 from 79.7468 % to 84.4842 %, with the advantage extending from top‑k 5 to 50, though it does not improve overall final‑answer F1.
By Shumao Sun