DiaLong is a new long‑context dialogue benchmark that tests transformer‑based Large Language Models’ ability to retain and recall memories. The dataset contains 100 human‑created conversations, expanded with LLMs to exceed lengths of existing datasets, and includes a fact‑recall task where models must classify 40 candidate facts as true or false. Experiments show current LLMs and memory‑enhancement methods still struggle, indicating a need for improved memory performance in extended conversations.
By Owen Bianchi, Samer Aslan, Ryunosuke Saito
arXiv:2608.22310v1 Announce Type: new
Abstract: Long-term memory is crucial for personalized responses and long-horizon agent interactions. Existing methods often rely on LLMs to compress or rewrite...
By Yuanhua Lin, Yile Li, Zhiyuan Zhao, Jing Shang, Jian Sun
PRAGMA is a benchmark designed to evaluate personalized guidance in long‑term conversations. It includes curated longitudinal conversation histories, evidence annotations, and guidance scenarios that reflect evolving user contexts and incorrect assumptions. Experiments show that current retrieval, memory, and long‑context models struggle to recover relevant conversational evidence and to use it effectively for personalized guidance.
By Hyojeong Yu, Hyukhun Koh, Minsung Kim, Yunah Jang, Kyomin Jung
The paper introduces LOCOMO-CONV, a conversational memory benchmark that expands on the existing LoCoMo dataset with four query styles—dialog, implicit, counterfactual, and composed—designed to evaluate memory systems in realistic conversational settings. Experiments across five memory systems reveal that conversational framing uncovers significant retrieval gaps missed by traditional QA benchmarks, particularly for implicit and composed queries, and that strong retrieval does not necessarily translate into higher response quality. The study also highlights silent grounding in implicit queries, where memory enhances contextual grounding without explicitly presenting the gold fact, suggesting a need for reasoning-based memory elaboration.
By Wen-Yu Chang, Yun-Nung Chen
UTILMEM is a new diagnostic benchmark that tests how conversational agents use long‑term memory, focusing on reasoning over dense histories, spotting implicitly relevant memories, synthesizing distributed evidence, and resisting interference from similar distractors. It contains 1,717 instances across five domains and evaluates a range of retrieval‑based and memory‑augmented systems. The study shows that strong performance on traditional factual recall does not guarantee effective memory utilization, highlighting a gap between retrieving information and integrating it into coherent, task‑oriented outputs.
By Peijun Qing, Fobo Shi, Soroush Vosoughi
Large language model agents have shown strong capabilities in generating coherent and contextually appropriate responses, yet robust long-horizon dialogue remains limited by the lack of external memory that is traceable, updatable, and diagnostically transparent. Existing memory-augmented agents often store memories as isolated records or overwritable states, making it difficult to preserve how information originates, evolves, conflicts, or becomes obsolete over time.
CueMem is a cue‑guided framework for long‑term conversational memory that reconstructs query‑relevant dialogue context from compressed memory records. Instead of treating memory units as self‑contained evidence, it extracts fine‑grained cues linked to their source turns and, at query time, expands from these cues over a turn graph to rebuild a compact evidence context. Experiments on LoCoMo and LongMemEval show that CueMem outperforms baseline memory methods, reduces input tokens and latency, and improves long‑term conversational question answering.
By Changjian Wang, Rongzhen Li, Weili Guan, Shuming Shi, Quan Lu, Ning Jiang
Scalaris is a scalable enterprise memory architecture designed to balance retrieval accuracy and context efficiency for conversational AI assistants. It separates user interactions and assistant responses into distinct long‑term and short‑term memory streams, each with specialized representations and ingestion/retrieval pipelines. The system achieves state‑of‑the‑art benchmark scores while reducing retrieved context tokens by about tenfold and improving overall response quality by 36 % in proprietary deployments.
By Devarsh Gandhi, Sarthak Pawar, Hemprasad Badgujar, Vaibhav Rathore, Siddharth Sureka, Neha Bhargava
arXiv:2606. 02461v2 Announce Type: replace Abstract: Language agents spend substantial inference time solving individual tasks, yet the experience acquired in one episode is often underutilized in future episodes.
By Yiheng Shu, Bernal Jim\'enez Guti\'errez, Saisri Padmaja Jonnalagedda, Yuguang Yao, Huan Sun, Yu Su
arXiv:2606. 02461v1 Announce Type: new Abstract: Language agents spend substantial inference time solving individual tasks, yet the experience acquired in one episode is often underutilized in future episodes.
By Yiheng Shu, Bernal Jim\'enez Guti\'errez, Saisri Padmaja Jonnalagedda, Yuguang Yao, Huan Sun, Yu Su
The paper introduces QGMem, a memory framework for conversational agents that organizes dialogue history into event-indexed atomic units and consolidates related units into dynamic memory traces. When a query is posed, QGMem retrieves complementary candidate memories, reranks them to form a compact working memory, and represents this working memory as a local graph to expose relational dependencies. Experiments on six benchmarks demonstrate consistent improvements in retrieval, multi‑hop evidence composition, conflict resolution, and ultra‑long dialogue reasoning while keeping context size and inference cost moderate.
By Yichen Liu, Chunfeng Yuan, Haowei Liu, Wenjuan Li, Zefeng Lin, Bing Li, Xu Chen, Weiming Hu
arXiv:2607. 12893v1 Announce Type: new Abstract: Long-term memory has become a foundational capability for LLM-based agents that accompany users across extended, multi-session interactions.
By Xixuan Hao, Zeyu Zhang, Zehao Lin, Yihang Sun, Ziliang Guo, Xichong Zhang, Yuxuan Liang, Feiyu Xiong, Zhiyu Li