← Back to all news
arXiv Computation and Language By Olga Ohrimenko

Eternal Sunshine of the Spotless Mind: Systematically Erasing LLM's Memories

Read the original on arXiv Computation and Language →

The paper investigates whether large language models (LLMs) that store persistent memories can truly forget information upon user request. It shows that existing LLMs cannot delete such memories even when they claim to have forgotten them, and that simply removing matching messages is ineffective due to message dependencies. The authors introduce DeLLM, a framework that builds relevant context dynamically and uses a provenance graph to identify which messages must be removed, achieving a high deletion rate while preserving utility.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv Machine Learning
Sep 14

AIM: A Privacy-Aware Interoperable Memory Framework for Multi-Agent Multi-User LLM Systems

The paper introduces AIM, a privacy‑aware memory framework that lets multi‑agent, multi‑user large language models manage both private and shared memory. AIM classifies data as private (user‑specific) or public (shared) and enforces index‑level access controls to protect sensitive information while enabling shared knowledge to improve coordination. The authors also present MUMBench, a new dataset for evaluating memory operations in multi‑user settings, and report high accuracy metrics for AIM on this benchmark.

By Zachary Johnson, Nigel Boachie Kumankumah, Somya Chatterjee, Tejas Sathyamurthi, Min Chen, Xinyi Alice Li, Xiao Wang, Emily Morgan Gelchie, Jessica Lin, Sadid A. Hasan, Sulaiman Vesal
llmsagentsbenchmarkssafety
More like this →
arXiv Machine Learning
Sep 11

Evaluating Memory Structure in LLM Agents

The paper introduces StructMemEval, a benchmark designed to assess how well large language model (LLM) agents can organize their long‑term memory rather than merely recall facts. It compiles tasks that humans typically solve by structuring knowledge—such as transaction ledgers, to‑do lists, and trees—and evaluates agents on these. Experiments show that simple retrieval‑augmented LLMs struggle with such organization tasks, while memory‑augmented agents perform better when explicitly prompted to structure their memory, yet many modern LLMs still fail to recognize memory structures without prompting.

By Alina Shutova, Alexandra Olenina, Ivan Vinogradov, Anton Sinitsin
llmsragagentsbenchmarks
More like this →