← Back to all news
arXiv Machine Learning By Xiaobing Yu, Peijie Qiu, Jin Yang, Xuanzhao Dong, Weiwei Ma, Zhaoqi An, Xiaoqi Zhao, Xiaofeng Liu

LadderEdit: Edit-Level Residual Compression for Memory-Efficient Lifelong Editing of LLMs

Read the original on arXiv Machine Learning →

LadderEdit is a method for memory‑efficient lifelong editing of large language models that compresses each LoRA adapter after acquisition. Each edit is initially stored as a low‑rank sketch and only promoted to higher rank if it fails to satisfy rewrite, generalization, and locality contracts on probe prompts. On benchmarks such as ZsRE, CounterFact, and WikiBigEdit, LadderEdit achieves the same performance as exact LoRA while using 5.2× less memory and handling up to 50,000 sequential edits.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Aug 28

LowRankArena: A Standardized Evaluation Platform for SVD-Based LLM Compression

LowRankArena is a standardized evaluation platform for SVD‑based low‑rank compression of large language models, unifying task versions, compression budgets, comparison regimes, and inference measurements. It provides a reproducible pipeline with over 3 TiB of released compressed checkpoints, enabling consistent comparisons across methods. An audit of five representative SVD techniques using LowRankArena shows that prior reported gains are highly conditional, with performance leaders and tiers shifting across backbones and keep ratios, and that nominal low‑rank savings often yield limited end‑to‑end speedups.

By Zishan Shao, Lixun Zhang, Kangning Cui, Wenhao Wu, Jinhee Kim, Yixiao Wang, Ting Jiang, Hancheng Ye, Qinsi Wang, Fan Yang, Danyang Zhuo, Yiran Chen, Hai Li
llmsbenchmarks
More like this →
arXiv AI
2d ago

Editing Large Language Models with Geometry-Aware Regularization

The paper introduces GEdit, a geometry-aware framework for editing large language models that leverages a Low-Rank Plus Diagonal factor model to capture the low-rank structure of Transformer FFN hidden representations. GEdit applies low‑rank regularization to protect core knowledge and adaptive regularization to adjust constraints based on the difficulty of each edit, allowing harder edits to succeed while preserving easier ones. Experiments on GPT2‑XL, GPT‑J, and LLaMA‑3‑8B with up to 10,000 continuous updates demonstrate that GEdit outperforms state‑of‑the‑art baselines in both editing success and knowledge preservation.

By Bingqing Liu, Wei Liu, Jun Wang, Xiaobo Sun, Yuhua Li, Ruixuan Li
llmsbenchmarks
More like this →