The paper investigates how tokenization can undermine post‑release guarantees that sensitive knowledge has been edited or unlearned from open‑weight large language models. By showing that alternative valid tokenizations can bypass localized modifications, the authors introduce Toketive, a reference‑free attack that detects modified knowledge and reconstructs pre‑edit responses using only the released model. Experiments on five LLMs, six datasets, and six editing techniques reveal that 38.6% of alternative tokenizations recover suppressed information, with Toketive achieving high detection and reconstruction accuracy.
By Manit Baser, Aditya Nawal, Dinil Mon Divakaran, Mohan Gurusamy
arXiv:2606. 00570v1 Announce Type: cross Abstract: Parameter-based knowledge editing updates the internal knowledge of large language models (LLMs) via localized weight modifications and has attracted significant attention.
By Wanying Ren, Xin Song, Futing Wang, Guoxiu He, Aixin Sun
The paper introduces GEdit, a geometry-aware framework for editing large language models that leverages a Low-Rank Plus Diagonal factor model to capture the low-rank structure of Transformer FFN hidden representations. GEdit applies low‑rank regularization to protect core knowledge and adaptive regularization to adjust constraints based on the difficulty of each edit, allowing harder edits to succeed while preserving easier ones. Experiments on GPT2‑XL, GPT‑J, and LLaMA‑3‑8B with up to 10,000 continuous updates demonstrate that GEdit outperforms state‑of‑the‑art baselines in both editing success and knowledge preservation.
By Bingqing Liu, Wei Liu, Jun Wang, Xiaobo Sun, Yuhua Li, Ruixuan Li
The paper introduces ALOE, a method for knowledge editing that learns semantic addresses from paraphrases and hard negatives, aligns them with autoregressive hidden states, and embeds a gated low‑rank operator within a single MLP layer. This design allows the edited model to run in one forward pass without external retrievers or routers. Experiments on CounterFact, ZSRE, and KnowEdit show high efficacy (0.955–0.999) and locality (0.981–1.000) across 7–8B model families, with analyses indicating effective separation of edits and suppression of out‑of‑scope activation.
By Zeyan Li, Hu Xu, Jianfeng Xu
arXiv:2606. 23276v2 Announce Type: replace Abstract: Knowledge Editing (KE) has emerged as a frontier for updating specific facts in LLMs without costly retraining, but its reliability and underlying mechanisms remain poorly understood.
By Advik Raj Basani, Anshuman Chhabra
The paper investigates whether knowledge editing truly erases original facts from language models. Using a linear trace probe, the authors find that after editing a fact in GPT‑2‑XL, the original object remains highly decodable from hidden states across three different editing methods, even when the model behaves correctly on edited prompts. This suggests that editing suppresses rather than removes the original association in representational space.
By Priyansh Srivastava, Romit Chatterjee
arXiv:2511. 05852v4 Announce Type: replace-cross Abstract: Knowledge editing (KE) offers a lightweight alternative to retraining for updating large language models (LLMs).
By Yinjie Cheng, Paul Youssef, Christin Seifert, J\"org Schl\"otterer, Zhixue Zhao
arXiv:2603.12677v4 Announce Type: replace-cross
Abstract: Existing locate-then-edit Knowledge Editing (KE) methods typically decompose editing into two stages: upstream target representation optimiza...
By Shuxin Liu, Di Gao, Ou Wu
The paper introduces a new white‑box attack on large language models that builds on knowledge‑editing techniques. By retrieving associative knowledge from the model, the attack extends constraint removal to a whole thematic category rather than just a predefined set of prompts. Experiments across different architectures show that this method improves attack effectiveness while preserving overall model performance.
By Roman Maksimov, Vladimir Aletov, Vladimir Solodkin, Dmitry Bylinkin, Daniil Medyakov, Aleksandr Beznosikov
arXiv:2602.20207v4 Announce Type: replace-cross
Abstract: Knowledge editing in Large Language Models (LLMs) aims to update the model's prediction for a specific query to a desired target while preser...
By Shrestha Datta, Hongfu Liu, Anshuman Chhabra
ROSE is a Risk‑Aware Orthogonal Subspace Navigation framework designed for lifelong knowledge editing in multimodal large language models. It limits parameter updates to the orthogonal complement of previously edited subspaces and uses a dynamic risk mask to protect critical factual parameters, thereby reducing catastrophic forgetting. During inference, ROSE applies prototype‑anchored semantic gating and perturbation‑driven subspace fusion to activate edited knowledge precisely for relevant queries while minimizing interference on unrelated inputs.
By Lingyun Song, Ziyao Chen, Kang Pan, Xiaolin Han, Yifei Zhang, Xiaoqi Wang, Yudai Pan, Xuequn Shang
arXiv:2607. 20433v1 Announce Type: cross Abstract: While language models remain frozen at their training state, the world evolves continuously.
By Jea Kwon, Jiwon Kim, Dong-kyum Kim, Meeyoung Cha