arXiv AI

MemQ: Integrating Q-Learning into Self-Evolving Memory Agents over Provenance DAGs

arXiv AI
Aug 25

CONTRAMEM: Learning Self-Evolving Procedural Memory from Contrasting Multi-Model Trajectories

arXiv:2608.22533v1 Announce Type: new Abstract: Autonomous computer-use agents are increasingly applied to long-horizon tasks requiring coordinated application calls, persistent state tracking, and v...

By Zheyuan Deng, Binghang Lu, Hanqi Feng, Shirley Huang, Dianzhuo Wang, Yuanda Xu, Zhiwei Zhang, Yige Sun, Changhong Mou, Runyu Zhang, Yuexing Hao, Barnabas Poczos, Xiaomin Li
arXiv AI
Sep 2

APEX-EM: Non-Parametric Online Learning for Autonomous Agents via Structured Procedural-Episodic Experience Replay

APEX-EM is a non‑parametric experience memory that stores full procedural‑episodic traces in a typed Procedural Knowledge Graph and retrieves them via semantic search, structural‑signature matching, and graph traversal. It uses a Plan‑Retrieve‑Generate‑Iterate‑Ingest workflow to produce, quality‑gate, and commit experiences, indexing both successes and failures so the agent learns what to reuse and what to avoid. Evaluations on five benchmarks with a shared GPT‑4o backbone show significant performance gains, such as +7.6 pp on BigCodeBench transfer and +1.4 pp on Lifelong Agent Bench, demonstrating that the memory adds to model capability rather than replacing it.

By Pratyay Banerjee, Masud Moshtaghi, Ankit Chadha
arXiv AI
Aug 3

Self-Play Meets Skill Evolution: Self-Evolving Search Agents that Pose, Solve, and Remember

arXiv:2607. 29468v1 Announce Type: new Abstract: Self-play agents can generate training problems without questions from target benchmarks, but their curricula lack persistent state: failures affect gradients yet do not explicitly shape future practice.

By Zenghuang Fu, Zhaoyang Li, Qiuyuan Ai, Haoyu Wu, Minghui Wu, Chenxu Zhao, Ante Wang, Guannan He, Changwei Wang
arXiv AI
Jul 16

Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agents

arXiv:2607. 13591v1 Announce Type: cross Abstract: Large Language Model (LLM) agents increasingly rely on external memory systems to accumulate experience across tasks.

By Eric Hanchen Jiang, Zhi Zhang, Yuchen Wu, Levina Li, Dong Liu, Xiao Liang, Rui Sun, Yubei Li, Edward Sun, Haozheng Luo, Zhaolu Kang, Aylin Caliskan, Kai-Wei Chang, Ying Nian Wu
arXiv AI
Oct 9

MemTrace: State-Consistent Memory for Long-Horizon Coding Agents

MemTrace is a provenance‑aware memory system designed to help coding agents handle long‑horizon software evolution tasks that span multiple files and stages. It stores execution history as immutable Memory Traces linked to key artifacts (files, symbols, tests) and organizes them in a Memory Trace Graph to track execution order and dependencies. When context budgets are limited, the system keeps only compact Memory Anchors, from which the agent can reconstruct the latest execution state and retrieve only valid, relevant evidence, improving performance on three long‑horizon coding benchmarks. "whyItMatters":"By preserving consistent task state and verifying evidence validity, MemTrace enables coding agents to outperform existing baselines on long‑horizon benchmarks, achieving significant gains in pass@1, resolved rate, and milestone scores."

By Hongming Xu, Le Zhou, ZhongHe Jin, Xiang Zhang, Bo Tang, Zhiyu Li, Xuanhe Zhou, Juncheng Zhang
arXiv AI
Jun 30

Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions

arXiv:2507. 05257v4 Announce Type: replace-cross Abstract: Recent benchmarks for Large Language Model (LLM) agents primarily focus on evaluating reasoning, planning, and execution capabilities, while another critical component-memory, encompassing how agents memorize, update, and retrieve long-term information-is under-evaluated due to the lack of benchmarks.

By Yuanzhe Hu, Yu Wang, Julian McAuley
Hugging Face Trending Papers
Aug 5

Caching for the Future: Scrub Jay Episodic Memory Principles for Agent Memory Systems

LLM agents that persist across sessions accumulate stored memories whose validity varies enormously by content type, yet existing memory architectures treat all memories as equally persistent and systematically contaminate retrieved context with outdated facts. We show that per-memory, type-conditioned temporal decay, a property of western scrub jay episodic memory, can be operationalized as an auto-classified coefficient $π_i$ in an external LLM-agent memory store, yielding ScrubJay-MEM: each memory is encoded as a jointly-bound What--Where--When tuple with an estimated perishability $π_i$ and utility horizon $τ_i$, retrieved by query-adaptive scoring, and revised retroactively at $O(1)$ LLM calls per update.

arXiv AI
Sep 3

CHIME: Credit-Aware Hierarchical Memory Evolution for Long-Horizon Agentic Planning

CHIME introduces a credit‑aware hierarchical memory evolution framework that separates planning and execution experiences into distinct memory banks. By attributing each task outcome to the plan, execution, both, or neither before memorization, CHIME mitigates bias from noisy final outcomes and improves long‑horizon agent planning. Experiments on four benchmarks demonstrate that CHIME outperforms existing training‑based and self‑evolving memory methods, requires fewer memory items, and transfers effectively across backbone models.

By Yongshi Ye, Tian Lan, Feihu Jiang, Muyang Ye, Bin Zhu, Qianghuai Jia, Longyue Wang, Zhao Xu, Weihua Luo, Xiaodong Shi