← Back to all news
arXiv Machine Learning By Syed Shariyar Murtaza, Yifan Nie, Utkarsh Soni, Eugene Wen, Arvid Frydenlund

When Synthetic Data Hurts: On Catastrophic Forgetting in Skill Retrieval for LLM Agents

Read the original on arXiv Machine Learning →

The paper investigates how synthetic data used for fine‑tuning large language model (LLM) skill routers can lead to catastrophic forgetting of real and out‑of‑distribution (OOD) skills. By evaluating a production router with 34,396 skills, the authors show that synthetic‑data fine‑tuning improves in‑distribution retrieval but degrades performance on real and OOD data. They test several continual‑learning inspired mitigation methods—embedding‑anchor regularization, Learning without Forgetting, Elastic Weight Consolidation, and L2‑initialization—and find that these approaches both preserve OOD retrieval performance and boost synthetic in‑distribution retrieval by up to 13.98% for a 0.6B Qwen retriever and reranker.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Oct 23

NADS: Navigator-Guided Data Selection for Mitigating Catastrophic Forgetting in Fine-Tuning

NADS (Navigator‑Guided Data Selection) is a fine‑tuning framework for large language models that mitigates catastrophic forgetting when the original pretraining data are unavailable. It first trains a navigator model without preservation regularization to detect capability drift, then scores synthetic data candidates using the predictive divergence between the navigator and the pretrained model. Finally, it selects a diverse, utility‑aware subset of synthetic data and fine‑tunes while distilling pretrained behavior, achieving a better balance between new‑task performance and general capability preservation with reduced selection overhead.

By Jianhao Zhang, Ou Wu, Yi Du
llmsfine-tuningsafety
More like this →
arXiv AI
Sep 28

Estimating and Orthogonalizing Unknown Pre-training Gradients for Continual Fine-tuning of Large Language Models

The paper introduces EoupCT, a framework that estimates and orthogonalizes unknown pre‑training gradients to mitigate catastrophic forgetting during continual fine‑tuning of large language models. It generates pseudo data most susceptible to forgetting using a learnable soft prompt with Gumbel‑Softmax, then jointly optimizes model parameters and the prompt via a first‑order Pareto optimizer to enforce orthogonality between new task updates and the estimated gradients. Experiments on multiple LLMs show that EoupCT preserves both task‑specific performance and the models’ inherent general‑purpose knowledge.

By Bing Wang, Changchun Li, Xin-Qiang Cai, Lin Yuanbo Wu, Ximing Li, Gang Niu, Masashi Sugiyama
llmsfine-tuning
More like this →
arXiv AI
Oct 2

Local Support Learning

arXiv:2610.02126v1 Announce Type: cross Abstract: We explore catastrophic forgetting in the context of large pre-trained models. By considering forgetting as a geometric problem in the input space of...

By Assaf Ben-Kish, Akarsh Kumar, James Glass, Raja Giryes
llms
More like this →