arXiv:2506. 14003v5 Announce Type: replace Abstract: Machine unlearning (MU) for large language models (LLMs), commonly referred to as LLM unlearning, seeks to remove specific undesirable data or knowledge from a trained model, while maintaining its performance on standard tasks.
The paper introduces LRE (Learned Relevance Eviction), a lightweight, CPU‑only, language‑model‑free scorer that learns which parts of an agent’s interaction history are task‑critical and preserves them verbatim. In experiments, LRE matches or surpasses baseline eviction policies on accuracy‑cost trade‑offs, recovers 93% of full‑history accuracy, reduces worst‑case prompt size by 52%, and outperforms dense and token‑pruning encoders in conversational memory while being 295–1569× smaller. The method also achieves superior budgeted answer quality on LoCoMo reading and can be trained annotation‑free, recovering 95% of supervised scorer performance.
The paper introduces GUARD, a method for natural forgetting in large reasoning models that transforms unsafe disclosures into safe-exit trajectories using guided answer‑reasoning distillation. It aligns a frozen model with guidance tokens and distills this behavior into the parameters, aiming for a coherent, non‑disclosing chain of thought followed by a refusal‑style answer. The authors also propose the Natural Forgetting Reasoning Score (NFRS) to evaluate structural stability, fluency, and unsupported substitutes, and demonstrate GUARD’s effectiveness on R‑TOFU and a STAR‑1‑derived harmful‑intent setting.
arXiv:2606. 17110v1 Announce Type: cross Abstract: Large Language Models are increasingly trained on proprietary or sensitive data, from private healthcare and financial records to user conversations containing secrets.
arXiv:2606. 10062v1 Announce Type: new Abstract: Foundation-model agents are increasingly long-lived systems that remember users across interactions, making memorization an explicit deployment-time function rather than solely a property of model weights.
arXiv:2610.07212v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) trains a language model on problems that may themselves be confidential, and the trained model ca...
arXiv:2610.28315v1 Announce Type: cross Abstract: Large language model (LLM) agents increasingly use persistent memory to personalize responses, and personal attributes in this memory pose a privacy...
The paper introduces JEPA‑Prune, a post‑hoc neuron‑pruning method that mitigates private data leakage in fine‑tuned large language models. It trains a public model on PII‑free surrogate views using a joint‑embedding predictive architecture, then uses cross‑model representation probes, integrated gradients, and a specificity score to identify and prune neurons that respond to personally identifiable information. Experiments show that pruning only 1.6% of neurons can dramatically reduce PII inference accuracy and recall while preserving model utility, outperforming existing privacy‑neuron editing techniques.
The paper introduces SparLeak, a side‑channel attack that exploits a new GPU micro‑architectural leakage called Sparsity‑Induced Memory Access (SIMA) caused by sparse attention in large language models. By capturing SIMA traces during LLM inference, SparLeak can infer user‑query attributes from prefill‑phase traces and reconstruct autoregressive responses from decoding‑phase traces. Experiments on three LLM architectures, three sparse attention mechanisms, and three privacy‑sensitive datasets show high success rates (90.9% for attribute inference and 87.3% for response reconstruction), underscoring the need to address SIMA leakage in sparse‑attention deployments.
arXiv:2511.05933v3 Announce Type: replace-cross Abstract: Reinforcement learning (RL) is often credited with improving reasoning at the expense of factual knowledge. We instead find that reasoning mo...
arXiv:2501.09166v2 Announce Type: replace-cross Abstract: Pretrained Transformers keep what they learned in their weights and lose what they observe once a session ends. The first version of this pap...
arXiv:2607. 09236v1 Announce Type: new Abstract: Machine unlearning in LLMs is the targeted removal of specific knowledge while preserving all other capabilities, critical for privacy and safety.