arXiv Computer Vision
Oct 19

Mitigating Task Objective Conflicts for Continual Learning in Unified Multimodal Models

The paper introduces UniDecouple, a selective decoupling framework for continual learning in unified multimodal models that addresses Task Objective Conflict (TOC). By employing a Task-Aware Mixture-of-Experts (TA‑MoE) architecture, it retains shared attention layers while adding task‑specific feed‑forward network experts to reduce optimization conflicts. Hierarchical expert routing and hybrid collaboration further balance task specialization with cross‑task knowledge sharing, and a two‑stage training strategy enhances both multimodal understanding and image generation performance.

By Jiaxing Zhang, Hao Tang
arXiv AI
Jul 3

Hidden Forgetting in Continual Multimodal Learning: When Accuracy Survives but Grounding Fails

arXiv:2607. 02020v1 Announce Type: new Abstract: Multimodal large language models must continually adapt to evolving tasks and domains, yet standard continual learning metrics mainly measure whether old answers remain correct, leaving the stability of multimodal grounding largely unexamined.

By Qianyu Chen, Canran Xiao, Runxuan Tang