Large language models (LLMs) have demonstrated strong capabilities in knowledge acquisition and reasoning, yet their ability to retain previously acquired knowledge under repeated updates remains insufficiently understood. Existing evaluation paradigms primarily focus on single-step reasoning or static knowledge editing, which fail to capture the temporal dynamics of knowledge retention and degradation during continual model modification.
The paper introduces GLIME, a lifelong model editing framework that integrates knowledge editing with preference optimization to handle continual updates in large language models. GLIME employs replay-based editing and a gradient constraint to prevent overfitting to target prompts and preserve previously edited knowledge. Experiments demonstrate that GLIME enhances knowledge generalization while maintaining editing performance and overall model capabilities.
By Dahyun Jung, Suhyune Son, Heuiseok Lim
arXiv:2603. 12658v2 Announce Type: replace-cross Abstract: Continual learning (CL) has emerged as a pivotal paradigm to enable large language models (LLMs) to dynamically adapt to evolving knowledge and sequential tasks while mitigating catastrophic forgetting, a critical limitation of the static pre-training paradigm inherent to modern LLMs.
By Hongyang Chen, Zhongwu Sun, Hongfei Ye, Kunchi Li, Xuemin Lin
arXiv:2607. 22556v1 Announce Type: new Abstract: Continual learning (CL) is essential for small language models (SLMs) to adapt to evolving real-world needs in resource-constrained deployments.
By Dong Li, Yanchi Liu, Xujiang Zhao, Wei Cheng, Zhengzhang Chen, Xintao Wu, Zhong Chen, Chen Zhao, Haifeng Chen
The paper introduces RD-Forget, a training‑free framework that separates what a persistent language agent stores from what it uses at answer time. It keeps a source archive of all observations while a query‑conditioned memory view filters evidence relevant to the current question, using a frozen language‑model curator to group facts into semantic slots and preserve multi‑hop relations. The approach employs rate‑distortion principles to stay within a memory budget and demonstrates improvements across conversational memory, knowledge updating, fact consolidation, long‑context reasoning, and personalization tasks.
By Yuhang Li, Yuchen Li
Persistent language agents need stored experience to remain available across time, while each answer requires evidence suited to a particular question. A superseded fact can mislead a current-state an...
arXiv:2511. 20892v4 Announce Type: replace Abstract: Large language models (LLMs) often produce incorrect or outdated content after being employed.
By Xuyuan Liu, Shengyu Chen, Xinshuai Dong, Yanchi Liu, Xujiang Zhao, Haoyu Wang, Yujun Yan, Haifeng Chen, Zhengzhang Chen
arXiv:2608. 11248v1 Announce Type: new Abstract: Long-term memory is essential for language agents operating across extended interactions and evolving tasks.
By Yuxi Qian, Yuxiang Ren
arXiv:2607. 09236v1 Announce Type: new Abstract: Machine unlearning in LLMs is the targeted removal of specific knowledge while preserving all other capabilities, critical for privacy and safety.
By Amit Peleg, Naman Deep Singh, Naama Pearl, Bibhabasu Mohapatra, Matthias Hein
arXiv:2606. 19679v1 Announce Type: cross Abstract: Lifelong knowledge editing aims to efficiently and sequentially update language models over time, as new knowledge becomes available or when the model makes mistakes, while preserving acceptable performance on past knowledge.
By Masih Eskandar, Miquel Sirera Perell\'o, Stratis Ioannidis, Jennifer Dy
arXiv:2607. 12893v1 Announce Type: new Abstract: Long-term memory has become a foundational capability for LLM-based agents that accompany users across extended, multi-session interactions.
By Xixuan Hao, Zeyu Zhang, Zehao Lin, Yihang Sun, Ziliang Guo, Xichong Zhang, Yuxuan Liang, Feiyu Xiong, Zhiyu Li
LLM agents increasingly rely on long-term memory to support multi-session interaction and personalization. However, existing agent memory systems are designed around forward-only evolution, continuously accumulating, consolidating, and overwriting knowledge, with no principled mechanism to inspect, version, or revert prior states.