arXiv AI
MetaKE: Meta-Learning for Knowledge Editing Toward a Better Accuracy-Editability Trade-off
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Aug 19
DOW-KE: Anchor-Free Multi-Layer Knowledge Editing via Direct End-to-End Weight Optimization
DOW-KE is an anchor‑free knowledge‑editing method that directly optimizes weight updates across multiple layers by backpropagating the final editing objective through the entire model. Unlike traditional locate‑then‑edit pipelines that optimize intermediate activations and then apply local weight updates—leading to a closure gap—DOW-KE ensures that what is optimized is exactly what is deployed, incorporating preservation constraints within the update parameterization. Experiments on two datasets and three models show that DOW-KE achieves the highest overall Score and neighborhood Specificity in five of six evaluated settings.
arXiv AI
Sep 30
Generalizable Lifelong Model Editing via Preference Optimization
The paper introduces GLIME, a lifelong model editing framework that integrates knowledge editing with preference optimization to handle continual updates in large language models. GLIME employs replay-based editing and a gradient constraint to prevent overfitting to target prompts and preserve previously edited knowledge. Experiments demonstrate that GLIME enhances knowledge generalization while maintaining editing performance and overall model capabilities.
arXiv AI
Sep 25
ALOE: Semantically Addressed Low-Rank Operators for Knowledge Editing
The paper introduces ALOE, a method for knowledge editing that learns semantic addresses from paraphrases and hard negatives, aligns them with autoregressive hidden states, and embeds a gated low‑rank operator within a single MLP layer. This design allows the edited model to run in one forward pass without external retrievers or routers. Experiments on CounterFact, ZSRE, and KnowEdit show high efficacy (0.955–0.999) and locality (0.981–1.000) across 7–8B model families, with analyses indicating effective separation of edits and suppression of out‑of‑scope activation.
arXiv AI
Jul 10
UltraX: Refining Pre-Training Data at Scale with Adaptive Programmatic Editing
arXiv:2607. 08646v1 Announce Type: cross Abstract: As available training data approaches its physical limit, gains from Scaling Laws have begun to diminish.
arXiv AI
Jun 16
Retrievable Gradients: Continual Post-Training Without Cumulative Weight Drift
arXiv:2606. 15734v1 Announce Type: cross Abstract: Continual post-training enables models to absorb emerging knowledge after deployment, but repeatedly updating shared parameters can accumulate weight drift, potentially causing catastrophic forgetting and degrading general capabilities.
arXiv AI
Jun 24
Representation Interventions Enable Lifelong Knowledge Memory Control in LLMs
arXiv:2511. 20892v4 Announce Type: replace Abstract: Large language models (LLMs) often produce incorrect or outdated content after being employed.