arXiv AI

MemTrace: Probing What Final Accuracy Misses in Long-Term Memory

arXiv:2606. 17328v1 Announce Type: new Abstract: LLM agents increasingly maintain long-term memory of user facts across sessions.

Hugging Face Trending Papers
Aug 20

MemTrapBench: Benchmarking Cognitive Traps in LLM Memory Use

Memory has become a key component of large language models, enabling them to retain information and learn from long-term interactions. However, existing memory benchmarks mainly evaluate whether information is correctly extracted, stored, and retrieved, while largely overlooking how retrieved memories reshape model reasoning and affect performance on the current task.

arXiv Computation and Language
Sep 1

UTILMEM: Benchmarking Evidence Utilization in Long-Term Conversational Memory

UTILMEM is a new diagnostic benchmark that tests how conversational agents use long‑term memory, focusing on reasoning over dense histories, spotting implicitly relevant memories, synthesizing distributed evidence, and resisting interference from similar distractors. It contains 1,717 instances across five domains and evaluates a range of retrieval‑based and memory‑augmented systems. The study shows that strong performance on traditional factual recall does not guarantee effective memory utilization, highlighting a gap between retrieving information and integrating it into coherent, task‑oriented outputs.

By Peijun Qing, Fobo Shi, Soroush Vosoughi
arXiv AI
Sep 10

When Does Memory Help? A Cost-Aware Evaluation of Long-Term Memory in Tool-Using LLM Agents

The paper introduces MERIT, a benchmark that evaluates the marginal benefit of long‑term memory for tool‑using large language model agents while explicitly accounting for cost. MERIT provides episodic tool‑use tasks across three domains, verifies dependence on earlier‑episode facts, and measures memory operations in tokens and dollars. Experiments on GPT‑4.1‑mini, Claude Haiku 4.5, and Claude Sonnet 5 show that memory can significantly improve task success, but its utility varies widely across models and memory implementations, and full replay is rarely cost‑effective.

By Shweta Mishra, Shashank Mishra
arXiv Computation and Language
4d ago

Memory Consolidation Flattens the Temporal Shape of User Facts

The paper introduces the concept of aspectual flattening, where long‑term memory systems convert conversational statements into concise notes that omit temporal cues. Using the LAPSE benchmark, the authors show that several memory writer models consistently flatten progressive statements while preserving simple‑present ones, and this asymmetry persists across multiple configurations. Experiments reveal that the loss of temporal information can alter how downstream readers assess the validity of a fact and influence their decision‑making, sometimes leading them to act without further verification.

By Sugam Panthi, Muhaiminul Yeamin, Siyan Luo, Rabab Abdelfattah
arXiv AI
4d ago

Memory Is a Derivation: The Distributed-Evidence Paradox in Long-Term Agents

The paper introduces the Distributed‑Evidence Paradox, where long‑running LLM agents compress past interactions into persistent memories that may not be fully supported by the interaction history. It defines three key requirements—evidence scope, compositional validity, and admission reliability—and proposes DerivAudit, a framework that checks whether a memory is truly supported by the available history. Experiments on two memory corpora show that expanding the evidence base can recover support for many memories, yet many remain unsupported, and broader evidence alone does not guarantee reliable admission.

By Hongjun Liu, Chen Zhao