arXiv AI By Hochan Son, Kyungdoe Han, Jaehan Koh, Xiaowu Dai, Wenlu Xu, Guang Cheng

Persistent Memory in Multi-Agent LLM Inference: What It Costs, What It Buys, and When You Can Tell

Read the original on arXiv AI →

The paper evaluates the impact of adding a persistent memory tier to a multi‑agent large language model inference system. While decomposing long‑context inference across agents reduces the peak KV cache usage from about 35 MiB to 14.3 MiB per query, the persistent tier adds only a modest 0.368 MiB to the cache and shows no measurable accuracy improvement across eight dataset pairs. The authors argue that the lack of benefit is structural, as single‑question benchmarks do not provide informative recall opportunities, and they outline conditions and detection procedures for agent‑memory ablations.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Machine Learning
Jul 10

What to Keep, What to Forget: A Rate--Distortion View of Memory Compaction in LLMs and Agents

arXiv:2607. 08032v1 Announce Type: new Abstract: Large language models, and the agents built on them, spend an ever-growing share of their compute and memory on remembering: caching attention keys and values, carrying long prompts, maintaining recurrent state, and storing what happened in previous turns and sessions.

By Ashwin Gerard Colaco, Nada Lahjouji
arXiv AI
1d ago

Understanding and Mitigating Inference-Time Overreliance Using Agentic Memory

The paper investigates how large language model agents can over-rely on agentic memory, a phenomenon where retrieved memories distort inference even when they are correctly stored and retrieved. It shows that memory is helpful when past experience fully transfers to the current task but becomes misleading under partial query-memory overlap, a pattern confirmed by controlled experiments. To address this, the authors propose MEMTRIM, a plug‑and‑play framework that indexes memory evidence at write time and limits its reuse at read time, thereby reducing over-reliance without retraining and preserving useful memory benefits across models and memory architectures.

By Luoxi Tang, Yuqiao Meng, Nilesh Auradkar, Muchao Ye, Dazheng Zhang, Zhaohan Xi
Hugging Face Trending Papers
Aug 5

Caching for the Future: Scrub Jay Episodic Memory Principles for Agent Memory Systems

LLM agents that persist across sessions accumulate stored memories whose validity varies enormously by content type, yet existing memory architectures treat all memories as equally persistent and systematically contaminate retrieved context with outdated facts. We show that per-memory, type-conditioned temporal decay, a property of western scrub jay episodic memory, can be operationalized as an auto-classified coefficient $π_i$ in an external LLM-agent memory store, yielding ScrubJay-MEM: each memory is encoded as a jointly-bound What--Where--When tuple with an estimated perishability $π_i$ and utility horizon $τ_i$, retrieved by query-adaptive scoring, and revised retroactively at $O(1)$ LLM calls per update.