arXiv AI
Aug 11

LoRSA: Toward Generalizable Parameter-Efficient Fine-Tuning for Biomedical Downstream Tasks

arXiv:2608. 07749v1 Announce Type: cross Abstract: Parameter-efficient fine-tuning enables the adaptation of vision foundation models to biomedical tasks under limited computational resources, but a single low-rank update can constrain all task-specific changes to one narrow parameter subspace.

By Saed Moradi, Benyamin Ghojogh, M. Hadi Sepanj, Yimin Yang, Ashirbani Saha
arXiv Computer Vision
Sep 14

Early Intervention for VFM-based Multimodal Medical Image Classification

The paper introduces an Early Intervention (EI) framework for multimodal medical image classification that addresses two key challenges: limited exploitation of complementary multimodal information and scarcity of labeled data for Vision Foundation Models (VFMs). EI treats one modality as the target and uses high‑level semantic tokens from other modalities as intervention tokens to guide the target’s embedding early in the process. The authors also propose Mixture of varied‑rank LoRAs (MoR) for efficient VFM adaptation, and demonstrate the method’s effectiveness on retinal, skin, and knee medical image datasets.

By Qijie Wei, Hailan Lin, Xirong Li