arXiv AI By Bocheng Ju, Jianhua Wang, Chengliang Liu, Xiaolin Chang

Null-Space Constrained Low-Rank Adaptation for Response-Specified Large Language Model Unlearning

Read the original on arXiv AI →

arXiv:2606. 10989v1 Announce Type: new Abstract: Large language model unlearning aims to suppress designated undesirable knowledge while preserving benign capabilities.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv Machine Learning
Jul 2

ZO-Act: Efficient Zeroth-Order Fine-Tuning via One-Shot Activation-Informed Low-Rank Subspaces

arXiv:2607. 01125v1 Announce Type: new Abstract: Zeroth-order (ZO) optimization enables fine-tuning large language models when backpropagation is unavailable or memory-prohibitive, but existing methods often perturb full model weights or randomly constructed low-dimensional subspaces, yielding high-variance estimates and limited performance.

By Xun Dong, Yibo Xu, Naigang Wang, Xin Li, Penghang Yin, Zi Yang