arXiv Computation and Language

Causal Episodic Memory for Feedback-Driven Agent Repair

The paper introduces MERIT, a training‑free agent that uses an online dual‑polarity memory of oracle‑verified corrections and failed attempts to improve LLM agent repair without updating parameters. MERIT retrieves only finalized memories, classifies failures, and employs a hybrid lexical‑dense retriever before the frozen model generates revisions. Experiments with Qwen2.5‑7B‑Instruct show that MERIT raises execution accuracy on the Spider benchmark from 66.34 % to 69.79 % and on BIRD from 47.35 % to 48.44 %, with varying evidence across datasets.

arXiv AI
Sep 10

When Does Memory Help? A Cost-Aware Evaluation of Long-Term Memory in Tool-Using LLM Agents

The paper introduces MERIT, a benchmark that evaluates the marginal benefit of long‑term memory for tool‑using large language model agents while explicitly accounting for cost. MERIT provides episodic tool‑use tasks across three domains, verifies dependence on earlier‑episode facts, and measures memory operations in tokens and dollars. Experiments on GPT‑4.1‑mini, Claude Haiku 4.5, and Claude Sonnet 5 show that memory can significantly improve task success, but its utility varies widely across models and memory implementations, and full replay is rarely cost‑effective.

By Shweta Mishra, Shashank Mishra
arXiv Computation and Language
Sep 22

EvalMem: An Operation-Level Diagnostic Framework for Long-Term Memory Systems

arXiv:2609.22231v1 Announce Type: new Abstract: Long-horizon interactions with LLM-based assistants require memory systems that preserve and update user states, preferences, and interaction histories...

By Zeyu Liu, Jian Zhong, Rongduo Han, Ziyang Wu, Shunye Tang, Chenghao He, Yaxuan Yang, Yihang Qiu, Ailing Wang, Xiao Liang, Guohuan Xie, Xiaokang Xue, Gongchen Li, Haining Zhang, Wei Wang
arXiv AI
Oct 16

Beyond Downstream QA: Rethinking How We Evaluate Agent Memory

The paper critiques current agent memory benchmarks that rely on downstream QA accuracy, arguing they conflate memory correctness with retrieval and generation quality, leading to misleading results. It introduces FEB-30, a new benchmark of 30 fact‑evolution trajectories across six diagnostic categories and three difficulty levels, generated automatically to provide deterministic ground truth. Using FEB-30, the authors show a significant divergence between intrinsic and extrinsic evaluations, revealing that a representative memory system correctly classifies memory operations only 20% of the time—an error invisible to traditional QA metrics.

By Sebastian-Antonio Toma, Jean de Kernier, Mike Ply, Oliviu Matei
arXiv AI
6d ago

Experience Memory Graph: Extracting Reusable Correction Knowledge from Agent Trajectories

The paper introduces Experience Memory Graph (EMG), a framework that transforms agent failure recovery into a graph matching problem. By converting failed and successful trajectories into directed action decision graphs, EMG extracts common subgraphs and edit paths that reveal explicit correction strategies, storing them in a memory graph with intra‑task nodes and cross‑task edges. During testing, EMG retrieves relevant insights to guide the agent in a single attempt, improving performance on ALFWorld and ScienceWorld without extra reflection rounds.

By Wenjun Wang, Yuchen Fang, Fengrui Liu, Zibo Liang, Kai Zheng
arXiv AI
Oct 21

REM: What to Store, When to Recall, What to Forget in Experience Memory for Coding Agents

The paper introduces REM (Reusable Experience Memory), a framework that manages the lifecycle of experience for large language model agents performing long-horizon coding tasks. REM consolidates completed trajectories into compact, phase-structured documents, injects them at task start, and evicts them after a single exposure, making the memory system responsive to the agent’s needs. Experiments on repository-level bug fixing show REM outperforms state‑of‑the‑art memory systems in retrieval accuracy and task resolution while reducing cost.

By Xinyi Lisa Chen, Pedro Herrero-Vidal, Xiaohan Chen, Xiaojing Yu, Cris Liu, Bingqing Song, Bin Xu, Suleiman A. Khan, You-Lin Chen
arXiv AI
Sep 4

Learning What Not to Forget: Long-Horizon Agent Memory from a Few Kilobytes of Learning

The paper introduces LRE (Learned Relevance Eviction), a lightweight, CPU‑only, language‑model‑free scorer that learns which parts of an agent’s interaction history are task‑critical and preserves them verbatim. In experiments, LRE matches or surpasses baseline eviction policies on accuracy‑cost trade‑offs, recovers 93% of full‑history accuracy, reduces worst‑case prompt size by 52%, and outperforms dense and token‑pruning encoders in conversational memory while being 295–1569× smaller. The method also achieves superior budgeted answer quality on LoCoMo reading and can be trained annotation‑free, recovering 95% of supervised scorer performance.

By Nusrat Jahan Lia, Aritra Mazumder
arXiv Computation and Language
Oct 19

ProPerAgent: Toward Memory-Driven Proactive Assistance

The paper introduces ProPerMem, a benchmark for memory-driven proactive assistance that pairs egocentric video with long, distractor-rich personal memories and human-verified trigger windows. Baseline models perform poorly, achieving at most 15% accuracy, while an oracle-driven GPT-5.3-Chat model reaches 96% when given evidence inside the trigger window. Building on these insights, the authors develop ProPerAgent, a plug‑and‑play system that retrieves relevant memory from a graph store and uses lightweight reinforcement learning to improve proactive intervention accuracy to 42.4% without training.

By Shenglai Zeng, Xiao Yang, Tiffany Ma, Xinyuan Zhang, Nanshu Wang, Deren Lei, Kai Sun, Xilun Chen, Tammy Stark, Xianhui Zhu, Yuetai Li, Rulin Shao, Tianxin Wei, Rakesh Wanga, Anuj Kumar, Jiliang Tang, Wen-tau Yih, Xin Luna Dong