arXiv Machine Learning
UnlearningSoup: Is Repeated Tuning Necessary for Large Language Model Unlearning?
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Jul 15
Inference-Time Machine Unlearning via Gated Activation Redirection
arXiv:2605. 12765v3 Announce Type: replace Abstract: Large Language Models memorize vast amounts of training data, raising concerns regarding privacy, copyright infringement, and safety.
arXiv Machine Learning
Jun 3
Fast Unlearning at Scale via Margin Self-Correction
arXiv:2606. 02920v1 Announce Type: new Abstract: Language-model unlearning updates a trained model to behave as if it had not seen selected training examples, while preserving utility and avoiding costly retraining.
arXiv AI
Aug 7
GROM: Gradient-Free Rapid One-Shot Machine Unlearning
arXiv:2608. 05783v1 Announce Type: cross Abstract: Machine unlearning has become a critical capability for safely removing specific, sensitive knowledge from large language models (LLMs).
arXiv Machine Learning
Jul 14
BalDRO: A Distributionally Robust Optimization based Framework for Large Language Model Unlearning
arXiv:2601. 09172v3 Announce Type: replace Abstract: As Large Language Models (LLMs) increasingly shape online content, removing targeted information from well-trained LLMs (also known as LLM unlearning) has become critical for web governance.
arXiv Computation and Language
Aug 25
CALIBURN: Self-Calibrated LLM Unlearning Alignment
CALIBURN is a new approach to large language model (LLM) unlearning that measures a model’s confidence in undesirable knowledge and uses this measure to fine‑tune unlearning gradient updates. By doing so, it offers more precise control over what is forgotten while better preserving the model’s overall utility. Experiments on benchmarks such as MUSE and WMDP show that CALIBURN outperforms existing methods in balancing knowledge removal with utility retention.
arXiv Machine Learning
Sep 10
RePro: Training Language Models to Faithfully Recycle the Web for Pretraining
RePro is a web‑recycling technique that trains a small language model (as little as 1 B parameters) with reinforcement learning to produce high‑quality, faithful rephrasings of pretraining data. The method uses one quality reward and three faithfulness rewards to preserve core semantics and structure while converting organic data into better training examples. Experiments show that RePro boosts downstream accuracy by 3.7–14.5 % over organic‑only baselines and improves data efficiency 2–3×, outperforming prior prompting‑based recycling approaches.