arXiv AI

AgentMemGate: Addressing Speculation Contamination in Conversational Assistant Memory

AgentMemGate is a write-time gate for conversational AI assistants that prevents speculation contamination in long‑term memory by classifying extracted statements as speculation, completed event, correction, or other. Speculations are kept outside memory until conditions allow promotion or deletion. The approach eliminates observed contamination and improves task accuracy from 65% to 95% on core benchmarks, with further gains on a harder held‑out set.

arXiv Computation and Language

Sep 1

UTILMEM: Benchmarking Evidence Utilization in Long-Term Conversational Memory

UTILMEM is a new diagnostic benchmark that tests how conversational agents use long‑term memory, focusing on reasoning over dense histories, spotting implicitly relevant memories, synthesizing distributed evidence, and resisting interference from similar distractors. It contains 1,717 instances across five domains and evaluates a range of retrieval‑based and memory‑augmented systems. The study shows that strong performance on traditional factual recall does not guarantee effective memory utilization, highlighting a gap between retrieving information and integrating it into coherent, task‑oriented outputs.

By Peijun Qing, Fobo Shi, Soroush Vosoughi

agents benchmarks

arXiv AI

Aug 5

LeanMem: Simple and Efficient Long-Term Memory for LLM Agents

arXiv:2608. 03463v1 Announce Type: new Abstract: Long-term memory is essential for LLM-based agents to sustain interactions and reliably leverage distant history.

By Yuxin Liao, Le Wu, Min Hou, Hao Liu, Han Wu, Zishu Wang

llms agents nlp

arXiv AI

Jun 12

MemRefine: LLM-Guided Compression for Long-Term Agent Memory

arXiv:2606. 13177v1 Announce Type: cross Abstract: Large language model (LLM) agents are increasingly expected to operate over long-term interactions, where information from past dialogues must be preserved and recalled to support future tasks.

By Minjae Kim, Jinheon Baek, Soyeong Jeong, Sung Ju Hwang

llms agents benchmarks

arXiv AI

5d ago

Gated Memory: Admission-Controlled Memory Formation for Conversational AI

The paper introduces Gated Memory, a framework for improving how conversational AI systems form long‑term memory. It adds two checkpoints before facts are stored: an admission gate that checks each candidate fact against the full utterance context, and a conditional enrichment stage that tags facts with provenance, scope, and time/place grounding. On the LoCoMo‑10 benchmark, Gated Memory improves LLM‑judge accuracy by 2.6% relative to a strong baseline.

By Preeti Saraswat, Divya Neelagiri, Ajay Manoj

llms benchmarks safety

arXiv AI

Jul 2

MemSyco-Bench: Benchmarking Sycophancy in Agent Memory

arXiv:2607. 01071v1 Announce Type: cross Abstract: Memory has emerged as a cornerstone of modern LLM-based agents, supporting their evolution from single-turn assistants to long-term collaborators.

By Zhishang Xiang, Zerui Chen, Yunbo Tang, Zhimin Wei, Ruqin Ning, Yujie Lin, Qinggang Zhang, Jinsong Su

llms agents benchmarks

arXiv AI

Sep 10

When Does Memory Help? A Cost-Aware Evaluation of Long-Term Memory in Tool-Using LLM Agents

The paper introduces MERIT, a benchmark that evaluates the marginal benefit of long‑term memory for tool‑using large language model agents while explicitly accounting for cost. MERIT provides episodic tool‑use tasks across three domains, verifies dependence on earlier‑episode facts, and measures memory operations in tokens and dollars. Experiments on GPT‑4.1‑mini, Claude Haiku 4.5, and Claude Sonnet 5 show that memory can significantly improve task success, but its utility varies widely across models and memory implementations, and full replay is rarely cost‑effective.

By Shweta Mishra, Shashank Mishra

llms rag agents nlp benchmarks

arXiv Machine Learning

Jul 15

Speculate with Memory: Lossless Acceleration for LLM Agents

arXiv:2607. 12236v1 Announce Type: new Abstract: Speculative execution accelerates LLM agents by using a smaller, cheaper model to predict and pre-launch the next step while the environment is idle.

By Yu Li, Qinyuan Ye, Prafulla Kumar Choubey, Jiaxin Zhang, Chien-Sheng Wu

llms agents benchmarks

Hugging Face Trending Papers

Jul 1

MemSyco-Bench: Benchmarking Sycophancy in Agent Memory

Memory has emerged as a cornerstone of modern LLM-based agents, supporting their evolution from single-turn assistants to long-term collaborators. However, memory is not always beneficial: retrieved memories often induce a critical issue of sycophancy, causing agents to over-align with the user at the cost of factual accuracy or objective reasoning.

llms agents benchmarks

arXiv AI

Jul 15

MemOps: Benchmarking Lifecycle Memory Operations in Long-Horizon Conversations

arXiv:2607. 12893v1 Announce Type: new Abstract: Long-term memory has become a foundational capability for LLM-based agents that accompany users across extended, multi-session interactions.

By Xixuan Hao, Zeyu Zhang, Zehao Lin, Yihang Sun, Ziliang Guo, Xichong Zhang, Yuxuan Liang, Feiyu Xiong, Zhiyu Li

llms agents nlp benchmarks

arXiv AI

Jul 8

Akashic: A Low-Overhead LLM Inference Service with MemAttention

arXiv:2607. 05708v1 Announce Type: new Abstract: Recent LLM-based agent systems continuously accumulate context across multi-turn interactions, tool invocations, and cross-session workflows.

By Yang Liu, Zhaokai Luo, Huayi Jin, Ruozhou He, Chenchen Hong, Zhiyong Wang, Yifei Liu, Yunfei Gu, Chentao Wu, Junhao Hu

llms agents efficiency

arXiv AI

Oct 7

MINDSET: Energy-based Schema Evolution for Long Conversational Agent Memory

arXiv:2610.08586v1 Announce Type: new Abstract: Long conversational agents have become essential in our daily lives. They must remember what was said long back in order to help us efficiently complet...

By Sujato Dutta, Sreekruthy Tummala, Shashank Vanga, Ayushmi Pavani

llms agents nlp

arXiv AI

Jun 16

EvoMemBench: Benchmarking Agent Memory from a Self-Evolving Perspective

arXiv:2605. 18421v2 Announce Type: replace-cross Abstract: Recent benchmarks for Large Language Model (LLM) agents mainly evaluate reasoning, planning, and execution.

By Yuyao Wang, Zhongjian Zhang, Mo Chi, Kaichi Yu, Yuhan Li, Miao Peng, Bing Tong, Chen Zhang, Yan Zhou, Jia Li

llms agents benchmarks

Related stories