The paper introduces a Fisher-guided submodular data selection method for continual pre‑training of large language models, addressing the forgetting problem that arises when a target‑domain corpus overwrites pretrained knowledge. By decomposing candidate gradients into anchor and frontier components and optimizing a log‑determinant submodular objective, the method selects data that both improves target‑domain performance and limits forgetting. Experiments on TinyLlama‑1.1B and Llama‑3.1‑8B show that the selector achieves superior adaptation and forgetting control while using ten times fewer tokens than traditional replay strategies.
By Zhenghao Zhao, Gaowen Liu, Zhiling Lan, Yan Yan
The paper investigates how medical vision‑language models (VLMs) behave when faced with distribution shifts such as changes in acquisition domain, supervision, or evaluation protocol. Using datasets like NIH ChestXray14, CheXpert, PadChest, and OpenI, the authors isolate cross‑dataset visual transfer, evaluate multimodal alignment, and quantify source‑proxy leakage in frozen embeddings. They find that self‑supervised visual initialization improves transfer, adversarial adaptation is only marginally helpful, and that multimodal retrieval performance drops under external stress tests while source‑proxy information remains recoverable, highlighting hidden failure modes in medical VLMs.
By Ayoub Louaye Bouaziz, Lokmane Chebouba, Yassine Himeur
arXiv:2608. 07749v1 Announce Type: cross Abstract: Parameter-efficient fine-tuning enables the adaptation of vision foundation models to biomedical tasks under limited computational resources, but a single low-rank update can constrain all task-specific changes to one narrow parameter subspace.
By Saed Moradi, Benyamin Ghojogh, M. Hadi Sepanj, Yimin Yang, Ashirbani Saha
arXiv:2607. 12048v1 Announce Type: cross Abstract: Deploying medical visual question answering (MedVQA) systems in real-world clinical settings requires models that adapt to new clinical tasks without forgetting previously acquired knowledge.
By Mai A. Shaaban, Tausifa Jan Saleem, Alaa Mohamed, Dilnaz Utemissova, Ufaq Khan, Mohammad Yaqub
arXiv:2410.21582v4 Announce Type: replace-cross
Abstract: Large-scale pretrained models are widely leveraged as foundations for learning new specialized tasks via fine-tuning, with the goal of mainta...
By Jaedong Hwang, Brian Cheung, Zhang-Wei Hong, Akhilan Boopathy, Pulkit Agrawal, Ila Fiete
arXiv:2610.09835v1 Announce Type: new
Abstract: Continual pre-training and fine-tuning in Large Language Models (LLMs) inevitably induce catastrophic forgetting, typically mitigated by replay using o...
By Jonghyun Han, Younghoon Song, Jongyoul Park
The paper evaluates federated learning with Low‑Rank Adaptation (LoRA) for fine‑tuning the BiomedCLIP vision‑language model on chest X‑ray classification across four international cohorts. Federated LoRA improves shared‑class AUC from 0.687 to 0.802, outperforming isolated single‑cohort training and approaching a centralized reference. The study shows that SVD‑based product‑space aggregation (FlexLoRA) is crucial for performance, while FedProx offers no advantage over FedAvg in this setting.
By Sanjaya Poudel, Nirajan Kunwor, Manish Dhakal, Debesh Jha, Sunil Kumar Gaire
arXiv:2609.21412v1 Announce Type: new
Abstract: Medical image segmenters often get worse when sites, scanner vendors, or protocols change. Continual test-time adaptation (CTTA) addresses this problem...
By Ruijie Huang
The paper introduces Foundation Preserving LoRA (FoLoRA), a forgetting‑aware optimization framework that balances adaptation to downstream tasks with preservation of pretraining capabilities. FoLoRA uses a first‑order preservation condition to define a forgetting penalty based on pretraining‑proxy activations and a task utility from downstream activations, scoring update directions via a generalized Rayleigh quotient. This spectral coordinate system enables gated Adam updates that reduce low‑utility, high‑penalty directions, and the method constructs pretraining proxy calibration data by sampling from the pretrained model. Experiments on math, code, and instruction‑following tasks demonstrate that FoLoRA achieves a stronger balance between target task performance and aggregate preservation of non‑target capabilities compared to baselines.
By Dongjun Kim, Adrian de Wynter, Huancheng Chen, Heasung Kim, Haris Vikalo
arXiv:2606. 00132v1 Announce Type: cross Abstract: While finetuning effectively adapts foundation models to specialized downstream tasks, it can degrade nontarget capabilities acquired during pretraining.
By Dongjun Kim, Adrian de Wynter, Huancheng Chen, Heasung Kim, Haris Vikalo
arXiv:2607. 05310v1 Announce Type: new Abstract: Model editing promises a fast, targeted way to correct post-deployment mistakes in medical vision-language models (VLMs) without costly retraining.
By Guli Zhu, Chenwei Wu, Liyue Shen
arXiv:2607. 12112v1 Announce Type: cross Abstract: Federated fine-tuning of Multimodal Large Language Models (MLLMs) across distributed networks enables privacy-sensitive adaptation to evolving data streams, yet a fundamental obstacle prevents robust deployment in dynamic environments: catastrophic forgetting, wherein sequential task updates erase previously acquired knowledge across visual, linguistic, and cross-modal representations.
By Jing Liu, Chenxuanyin Zou, Jiayang Ren, Gaoyun Fang, Chengfang Li, Yan Wang, Zhenchao Ma, Bo Hu