arXiv Machine Learning

Neural Succession: A Mesoscopic Theory of Invasion, Coexistence, and Stabilization in Continual Learning

arXiv AI
Sep 16

ReDraft, Don't Just Distill: Reference-Driven Revision for Continual VLLM Post-Training

ReDraft is a reference‑driven revision method for continual post‑training of large multimodal language models. It uses the model’s own incorrect outputs as references, revises them, verifies the revisions, and fine‑tunes on the accepted ones, thereby combining explicit supervision with policy proximity. On tasks such as Counting, Clock Reading, and Jigsaw, ReDraft outperforms standard supervised fine‑tuning and on‑policy methods, achieving higher target‑task gains while dramatically reducing forgetting.

By Zhihao Zhang, Mingqi Wu, Qiaole Dong, Enyu Zhou, Shuo Li, Boyang Liu, Jiazheng Zhang, Honglin Guo, Xin Guo, Shaofan Liu, Junzhe Wang, Dingwei Zhu, Zhiheng Xi, Minlong Peng, Yuan Hua, Qi Zhang, Tao Gui, Xuanjing Huang
arXiv Machine Learning
Oct 9

From a Prompt to Repertoires: Evolving Functional REpertoires Enable LLM Continual Learning

The paper introduces Evolving Functional REpertoires (EFRE), a method that replaces a single prompt with a dynamic set of functions to enable large language models to learn new tasks sequentially without forgetting. EFRE refines existing functions when updates are compatible and creates new ones when updates conflict, thereby mitigating catastrophic forgetting and overfitting. Experiments on a three‑task continual‑learning stream show EFRE outperforms GRPO by 7.50 percentage points and reduces performance drop on a subsequent task by only 1.56 percentage points compared to base prompt optimization.

By Fengyuan Liu, Yue Wang, Hangxi Guo, Fengyuan Liu, Chenxu Wu, Yanguang Liu, Mengnan Du
arXiv Machine Learning
Oct 23

How to Teach Large Multimodal Models New Skills

The paper investigates how to add new skills to large multimodal models (LMMs) without erasing previously learned abilities. By sequentially fine‑tuning on five target skills and tracking performance on eight held‑out benchmarks across three model families, the authors discover that forgetting on one skill can partially recover when the model is later tuned on a different skill. They attribute this to shifts in the output token distribution and propose two simple, selective tuning recipes—updating only self‑attention projection layers or only the MLP Gate&Up while freezing the Down projection—that achieve a better learning‑forgiveness trade‑off than full‑LLM tuning or other common forgetting‑mitigation methods. whyItMatters":"The study shows that carefully choosing which model components to fine‑tune can enable LMMs to acquire new skills while preserving prior knowledge, offering a simpler and more effective alternative to existing forgetting‑mitigation techniques."

By Zhen Zhu, Yiming Gong, Yao Xiao, Yaoyao Liu, Derek Hoiem