The paper introduces LOCOMO-CONV, a conversational memory benchmark that expands on the existing LoCoMo dataset with four query styles—dialog, implicit, counterfactual, and composed—designed to evaluate memory systems in realistic conversational settings. Experiments across five memory systems reveal that conversational framing uncovers significant retrieval gaps missed by traditional QA benchmarks, particularly for implicit and composed queries, and that strong retrieval does not necessarily translate into higher response quality. The study also highlights silent grounding in implicit queries, where memory enhances contextual grounding without explicitly presenting the gold fact, suggesting a need for reasoning-based memory elaboration.
By Wen-Yu Chang, Yun-Nung Chen
AI systems increasingly need to combine two demanding capabilities: navigating multi-session conversation history and performing deep reading comprehension within long documents. Yet no existing benchmark evaluates both simultaneously.
arXiv:2606. 04442v1 Announce Type: cross Abstract: AI systems increasingly need to combine two demanding capabilities: navigating multi-session conversation history and performing deep reading comprehension within long documents.
By Qiyang Xie, Jialun Wu, Xinjie He, Su Liu, Shuai Xiao, Zhiyuan Lin, Weikai Zhou
arXiv:2606. 28349v1 Announce Type: cross Abstract: Long-context reasoning requires models to access, retrieve, and integrate evidence scattered across documents, dialogues, and accumulated interaction histories.
By Zeju Li, Ziyang Zheng, Yizhou Zhou, Qiang Xu
CueMem is a cue‑guided framework for long‑term conversational memory that reconstructs query‑relevant dialogue context from compressed memory records. Instead of treating memory units as self‑contained evidence, it extracts fine‑grained cues linked to their source turns and, at query time, expands from these cues over a turn graph to rebuild a compact evidence context. Experiments on LoCoMo and LongMemEval show that CueMem outperforms baseline memory methods, reduces input tokens and latency, and improves long‑term conversational question answering.
By Changjian Wang, Rongzhen Li, Weili Guan, Shuming Shi, Quan Lu, Ning Jiang
The paper introduces MemCompile, a query‑conditioned evidence compilation framework designed to improve long‑term conversational memory. It uses an Address Plane to locate relevant support via compact descriptors and capacity‑aware cells, then normalizes retrieval hits into stable SourcePacket identities. An Evidence Plane allocates a bounded evidence budget across packets, admits limited additional support, and produces answer‑ready context, achieving 80.52% Judge Accuracy on LoCoMo and improving the quality‑cost Pareto frontier.
By Shangze Li, Yanyu Yao, Derong Xu, Zhi Zheng, Tong Xu
JustMem is a new memory system for long‑term conversational AI that stores conversation history as compact atomic memories and adapts its access strategy to each query. It introduces two dimensions of memory access—discovery breadth and reading fidelity—implemented through LOOKUP for local evidence, COMPOSE for distributed evidence, and REPLAY for fidelity‑sensitive evidence. Experiments on LoCoMo and LongMemEval‑S show that JustMem outperforms competing memory systems in accuracy and recall while using fewer generative‑model tokens for memory construction and inference.
By Guanhua Chen, Yanting Wang, Wenjing Zhi, Lei Sha
The paper introduces SCALE-QA, a new QA benchmark that tests conversational memory in flat, unsegmented multi‑topic threads by requiring agents to infer which earlier episode supports a later task decision. The dataset contains 3,000 audited questions across ten domains, uses deterministic four‑way multiple‑choice grading, and includes a runtime builder for reproducibility. The authors also propose Temporal‑Semantic Interleaved Memory Reconstruction (TSIM), a hierarchical memory stack that segments turns into coherent episodes and indexes them with deterministic summaries and cluster‑routing views, achieving significant accuracy gains over strong RAG baselines and long‑context LLMs.
By Zhexi Feng, Ruiyi Zhang, Yongbo Yang, Pengtao Xie
UTILMEM is a new diagnostic benchmark that tests how conversational agents use long‑term memory, focusing on reasoning over dense histories, spotting implicitly relevant memories, synthesizing distributed evidence, and resisting interference from similar distractors. It contains 1,717 instances across five domains and evaluates a range of retrieval‑based and memory‑augmented systems. The study shows that strong performance on traditional factual recall does not guarantee effective memory utilization, highlighting a gap between retrieving information and integrating it into coherent, task‑oriented outputs.
By Peijun Qing, Fobo Shi, Soroush Vosoughi
APDMem is a hierarchical long‑term memory system for personalized LLM assistants that uses progressive disclosure to retrieve conversation history. It organizes memory into four layers—thematic summaries, personalized key facts, turn‑level evidence notes, and raw messages—and a controller reads from the top layer, drilling down only when necessary. This approach balances cost and fidelity, allowing simple queries to finish early while deeper inspection is triggered for complex or exact‑evidence requests, and a note synthesizer structures retrieved evidence before answer generation. Experiments on LongMemEval show APDMem performs strongly while accessing only 8% of the total conversations.
By Chin-Lun Fu, Anagha Kulkarni, Hong Ni, Behrouz Madahian
arXiv:2608.12990v2 Announce Type: replace
Abstract: Long-horizon LLM agents must preserve information from past interactions to support future tasks. Existing memory systems typically rely on eager c...
By Dongfang Li, Zixuan Liu, Junmai Wang, Jiahe Huang, Fuhao Li, Lin Zhang, Bonian Jia, Baotian Hu, Min Zhang
arXiv:2607. 24097v1 Announce Type: new Abstract: Memory-augmented LLM agents typically answer queries by retrieving relevant memories and feeding them directly to an answer model.
By Yiwen Ma, Songjun Tu, Qichao Zhang, Dong Li, Linjing Li, Dongbin Zhao