arXiv AI

Fidelity Before Structure: Verbatim Chunks Beat Lossy Artifact Extraction in Long-Conversation LLM Memory

arXiv:2601. 00821v4 Announce Type: replace Abstract: A growing class of conversational-memory systems compresses dialogue history into structured artifacts (extracted facts, decisions, or events) on the premise that distilled structure retrieves better than raw text.

arXiv AI
Aug 26

RENDER: Controlling Reader-Facing Evidence in LLM Memory Evaluation

RENDER is a benchmark that controls the reader‑facing artifact in memory and RAG evaluations while keeping the conversation fixed. It introduces a five‑level packet ladder and deterministic templates that mimic ChatGPT‑style entries, LangChain summaries, MemGPT‑style typed records, and raw conversation. Experiments on 500 LongMemEval questions across nine models show that matched‑budget packets outperform raw dialogue by 42.4–72.6 points, and that ChatGPT‑style entries often score higher than raw conversation, with effects persisting under retrieval noise and transferring to HotpotQA.

By Yuan Si, Simeng Han, Daming Li, Jialu Zhang
arXiv Computation and Language
Aug 31

Entity-Memory Graph Retrieval Improves Evidence Coverage in Long-Conversation Question Answering

Entity-Memory graph retrieval preserves dialogue turns as verbatim memory nodes, links repeated mentions via shared entities, and connects adjacent memories with chronological edges. During retrieval, the system gates through entities, fuses semantics, and performs one‑hop chronological recovery before dense backfill, allowing it to keep neighboring memories that dense cosine ranking might miss. On 1,986 questions from ten LoCoMo conversations, this graph retrieval method increases official evidence recall at top‑k 25 from 79.7468 % to 84.4842 %, with the advantage extending from top‑k 5 to 50, though it does not improve overall final‑answer F1.

By Shumao Sun
arXiv Computation and Language
Sep 18

JustMem: Just-Enough Memory Access for Long-Term Conversations

JustMem is a new memory system for long‑term conversational AI that stores conversation history as compact atomic memories and adapts its access strategy to each query. It introduces two dimensions of memory access—discovery breadth and reading fidelity—implemented through LOOKUP for local evidence, COMPOSE for distributed evidence, and REPLAY for fidelity‑sensitive evidence. Experiments on LoCoMo and LongMemEval‑S show that JustMem outperforms competing memory systems in accuracy and recall while using fewer generative‑model tokens for memory construction and inference.

By Guanhua Chen, Yanting Wang, Wenjing Zhi, Lei Sha
arXiv AI
6d ago

HasMem: Hard-Origin Adaptively Softened Memory for Long-Term LLM Agents

The paper introduces Hard-Origin Adaptively Softened Memory (HasMem), a memory system for large language model agents that combines frozen hard‑prompt embeddings with a controller, writer, reader, and global module to adaptively resize and re‑encode memory entries. On a reconstruction probe of 535 questions, HasMem achieves a lexical F1 of 95.3, outperforming the hard reference by 4.4 percentage points while maintaining 93.6% of the reference’s memory positions. Across six configurations with similar per‑question budgets, the system surpasses rule‑based re‑encoding by 8.0–23.6 exact‑match points, and on LongMemEval‑S it improves local lexical F1 from 3.4 to 8.9 and reduces answer negative log‑likelihood from 12.257 to 5.274.

By Zihong He, Junxiao Shen, Chen Liang, Hai-Ning Liang
arXiv AI
Sep 4

When Users Don't Ask: Benchmarking Context-Driven Memory Retrieval in Conversational Agents

The paper introduces LOCOMO-CONV, a conversational memory benchmark that expands on the existing LoCoMo dataset with four query styles—dialog, implicit, counterfactual, and composed—designed to evaluate memory systems in realistic conversational settings. Experiments across five memory systems reveal that conversational framing uncovers significant retrieval gaps missed by traditional QA benchmarks, particularly for implicit and composed queries, and that strong retrieval does not necessarily translate into higher response quality. The study also highlights silent grounding in implicit queries, where memory enhances contextual grounding without explicitly presenting the gold fact, suggesting a need for reasoning-based memory elaboration.

By Wen-Yu Chang, Yun-Nung Chen