← Back to all news
arXiv Computation and Language By Lingyun Song, Ziyao Chen, Kang Pan, Xiaolin Han, Yifei Zhang, Xiaoqi Wang, Yudai Pan, Xuequn Shang

ROSE: Risk-Aware Orthogonal Subspace Navigation for Lifelong Knowledge Editing in Multimodal Large Language Models

Read the original on arXiv Computation and Language →

ROSE is a Risk‑Aware Orthogonal Subspace Navigation framework designed for lifelong knowledge editing in multimodal large language models. It limits parameter updates to the orthogonal complement of previously edited subspaces and uses a dynamic risk mask to protect critical factual parameters, thereby reducing catastrophic forgetting. During inference, ROSE applies prototype‑anchored semantic gating and perturbation‑driven subspace fusion to activate edited knowledge precisely for relevant queries while minimizing interference on unrelated inputs.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv AI
Jun 19

LOKI: Memory-Free Null-Space Constrained Lifelong Knowledge Editing

arXiv:2606. 19679v1 Announce Type: cross Abstract: Lifelong knowledge editing aims to efficiently and sequentially update language models over time, as new knowledge becomes available or when the model makes mistakes, while preserving acceptable performance on past knowledge.

By Masih Eskandar, Miquel Sirera Perell\'o, Stratis Ioannidis, Jennifer Dy
llms
More like this →
arXiv AI
Oct 5

Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing

The paper introduces FOVEATED, a plug‑and‑play framework that improves atomic‑fact recall in unstructured knowledge editing (UKE) for large language models. By randomly shifting Rotary Position Embedding (RoPE) positions during editing, FOVEATED creates focused views of each sentence, counteracting the context‑reliance problem where edited LLMs reproduce passages but fail to recall individual facts. Experiments show consistent gains across five editors, two LLM backbones, and three benchmarks.

By Ding Wu, Ye Zhang, Haoyu Wang, Tianci Liu
llmsragbenchmarks
More like this →
arXiv Computation and Language
Oct 8

EngramEdit: Decoupled Knowledge Updates in LLMs through Conditional Memory

EngramEdit introduces a method for decoupled knowledge updates in large language models using conditional memory architectures. By computing target memory representations that align with updated facts across multiple expressions, it jointly adjusts shared n‑gram embeddings while penalizing frequent ones to preserve unrelated knowledge. Experiments demonstrate near‑perfect editing success, improved multi‑hop reasoning, and strong performance under chain‑of‑thought prompting, all while maintaining general capabilities.

By Hongru Cai, Ran Wei, Wenjie Wang, Chengfa Wu, Ning Song, Yongqi Li, Wenjie Li
llmsrag
More like this →
arXiv AI
Oct 27

DynEdit: Dynamic Entropy-Guided Sequential Editing for Large Language Models

DynEdit is a dynamic framework for editing long-sequence knowledge in large language models. It uses token-level entropy to guide non-uniform sliding windows, focusing on high-entropy, information-dense regions that are harder to edit. Experiments on three benchmarks and two backbone models show DynEdit outperforms baselines, achieving up to +14.0 BLEU points on challenging paraphrase queries and near-saturated semantic scores across diverse knowledge domains.

By Jinhu Fu, Yan Bai, Yihang Lou, Li Sun, Jiahong Wu, Xiangxiang Chu, Sen Su
llmsbenchmarks
More like this →