arXiv AI
Jun 24
Representation Interventions Enable Lifelong Knowledge Memory Control in LLMs
arXiv:2511. 20892v4 Announce Type: replace Abstract: Large language models (LLMs) often produce incorrect or outdated content after being employed.
ROSE: Risk-Aware Orthogonal Subspace Navigation for Lifelong Knowledge Editing in Multimodal Large Language Models
ROSE is a Risk‑Aware Orthogonal Subspace Navigation framework designed for lifelong knowledge editing in multimodal large language models. It limits parameter updates to the orthogonal complement of previously edited subspaces and uses a dynamic risk mask to protect critical factual parameters, thereby reducing catastrophic forgetting. During inference, ROSE applies prototype‑anchored semantic gating and perturbation‑driven subspace fusion to activate edited knowledge precisely for relevant queries while minimizing interference on unrelated inputs.
Related stories
arXiv AI
Jun 19
LOKI: Memory-Free Null-Space Constrained Lifelong Knowledge Editing
arXiv:2606. 19679v1 Announce Type: cross Abstract: Lifelong knowledge editing aims to efficiently and sequentially update language models over time, as new knowledge becomes available or when the model makes mistakes, while preserving acceptable performance on past knowledge.
arXiv AI
Jul 3
Multimodal Knowledge Edit-Scoped Generalization for Online Recursive MLLM Editing
arXiv:2607. 01978v1 Announce Type: new Abstract: Online multimodal knowledge editing requires injecting a continual stream of visual-textual corrections into multimodal large language models (MLLMs) with bounded overhead and minimal disruption to unrelated behaviors.
arXiv AI
Oct 5
Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing
The paper introduces FOVEATED, a plug‑and‑play framework that improves atomic‑fact recall in unstructured knowledge editing (UKE) for large language models. By randomly shifting Rotary Position Embedding (RoPE) positions during editing, FOVEATED creates focused views of each sentence, counteracting the context‑reliance problem where edited LLMs reproduce passages but fail to recall individual facts. Experiments show consistent gains across five editors, two LLM backbones, and three benchmarks.
arXiv Computation and Language
Oct 8
EngramEdit: Decoupled Knowledge Updates in LLMs through Conditional Memory
EngramEdit introduces a method for decoupled knowledge updates in large language models using conditional memory architectures. By computing target memory representations that align with updated facts across multiple expressions, it jointly adjusts shared n‑gram embeddings while penalizing frequent ones to preserve unrelated knowledge. Experiments demonstrate near‑perfect editing success, improved multi‑hop reasoning, and strong performance under chain‑of‑thought prompting, all while maintaining general capabilities.
arXiv AI
Oct 27
DynEdit: Dynamic Entropy-Guided Sequential Editing for Large Language Models
DynEdit is a dynamic framework for editing long-sequence knowledge in large language models. It uses token-level entropy to guide non-uniform sliding windows, focusing on high-entropy, information-dense regions that are harder to edit. Experiments on three benchmarks and two backbone models show DynEdit outperforms baselines, achieving up to +14.0 BLEU points on challenging paraphrase queries and near-saturated semantic scores across diverse knowledge domains.
arXiv AI
4d ago
Editing Large Language Models with Geometry-Aware Regularization
The paper introduces GEdit, a geometry-aware framework for editing large language models that leverages a Low-Rank Plus Diagonal factor model to capture the low-rank structure of Transformer FFN hidden representations. GEdit applies low‑rank regularization to protect core knowledge and adaptive regularization to adjust constraints based on the difficulty of each edit, allowing harder edits to succeed while preserving easier ones. Experiments on GPT2‑XL, GPT‑J, and LLaMA‑3‑8B with up to 10,000 continuous updates demonstrate that GEdit outperforms state‑of‑the‑art baselines in both editing success and knowledge preservation.
arXiv AI
Jun 2
AnyEdit++: Adaptive Long-Form Knowledge Editing via Bayesian Surprise
arXiv:2606. 01053v1 Announce Type: new Abstract: Editing complex, long-form knowledge in Large Language Models remains a significant challenge due to the difficulty of maintaining generation coherence.
arXiv AI
Jun 8
Sparse Subspace-to-Expert Sharing for Task-Agnostic Continual Learning
arXiv:2606. 07500v1 Announce Type: cross Abstract: Continual learning in Large Language Models (LLMs) is hindered by the plasticity-stability dilemma, where acquiring new capabilities often leads to catastrophic forgetting of previous knowledge.
arXiv AI
Jun 10
Null-Space Constrained Low-Rank Adaptation for Response-Specified Large Language Model Unlearning
arXiv:2606. 10989v1 Announce Type: new Abstract: Large language model unlearning aims to suppress designated undesirable knowledge while preserving benign capabilities.
arXiv AI
Sep 25
ALOE: Semantically Addressed Low-Rank Operators for Knowledge Editing
The paper introduces ALOE, a method for knowledge editing that learns semantic addresses from paraphrases and hard negatives, aligns them with autoregressive hidden states, and embeds a gated low‑rank operator within a single MLP layer. This design allows the edited model to run in one forward pass without external retrievers or routers. Experiments on CounterFact, ZSRE, and KnowEdit show high efficacy (0.955–0.999) and locality (0.981–1.000) across 7–8B model families, with analyses indicating effective separation of edits and suppression of out‑of‑scope activation.
arXiv AI
Jul 24
Moir: Let the Model Direct Its Own Story for Robust Cross-Domain Knowledge Editing
arXiv:2607. 20433v1 Announce Type: cross Abstract: While language models remain frozen at their training state, the world evolves continuously.