arXiv AI By Zhihao Zhang, Mingqi Wu, Qiaole Dong, Enyu Zhou, Shuo Li, Boyang Liu, Jiazheng Zhang, Honglin Guo, Xin Guo, Shaofan Liu, Junzhe Wang, Dingwei Zhu, Zhiheng Xi, Minlong Peng, Yuan Hua, Qi Zhang, Tao Gui, Xuanjing Huang

ReDraft, Don't Just Distill: Reference-Driven Revision for Continual VLLM Post-Training

Read the original on arXiv AI →

ReDraft is a reference‑driven revision method for continual post‑training of large multimodal language models. It uses the model’s own incorrect outputs as references, revises them, verifies the revisions, and fine‑tunes on the accepted ones, thereby combining explicit supervision with policy proximity. On tasks such as Counting, Clock Reading, and Jigsaw, ReDraft outperforms standard supervised fine‑tuning and on‑policy methods, achieving higher target‑task gains while dramatically reducing forgetting.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Hugging Face Trending Papers
Oct 5

Off-Policy Merging Beats On-Policy Self-Distillation for Continual Learning

The paper demonstrates that off‑policy merging, called grafting, outperforms on‑policy self‑distillation for continual learning. Grafting learns updates on an earlier donor checkpoint, scales the update, and optionally masks sensitive directions, thereby reducing interference with existing capabilities. Across various continual learning scenarios, grafting achieves better new‑task and old‑task performance without costly on‑policy sampling.

arXiv Machine Learning
Oct 23

How to Teach Large Multimodal Models New Skills

The paper investigates how to add new skills to large multimodal models (LMMs) without erasing previously learned abilities. By sequentially fine‑tuning on five target skills and tracking performance on eight held‑out benchmarks across three model families, the authors discover that forgetting on one skill can partially recover when the model is later tuned on a different skill. They attribute this to shifts in the output token distribution and propose two simple, selective tuning recipes—updating only self‑attention projection layers or only the MLP Gate&Up while freezing the Down projection—that achieve a better learning‑forgiveness trade‑off than full‑LLM tuning or other common forgetting‑mitigation methods. whyItMatters":"The study shows that carefully choosing which model components to fine‑tune can enable LMMs to acquire new skills while preserving prior knowledge, offering a simpler and more effective alternative to existing forgetting‑mitigation techniques."

By Zhen Zhu, Yiming Gong, Yao Xiao, Yaoyao Liu, Derek Hoiem