arXiv Computer Vision
When Adaptation Hurts: Connecting Representational Drift to OOD Failures in MedSAM Fine-Tuning
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv Computer Vision
Sep 17
Lumen: Parameter-Efficient Alignment of Pretrained Vision and Language Encoders for Zero-Shot Computational Pathology
Lumen is a pathology vision‑language model that aligns frozen unimodal foundation models (Virchow2 and BioMedBERT) using rank‑4 adapters and projection heads, training only 0.40% of the total parameters on the QUILT‑1M corpus. It achieves the highest mean chance‑corrected balanced accuracy (0.546) across nine zero‑shot patch benchmarks and demonstrates strong performance on lymph‑node metastasis detection, with AUROC scores of 0.964 internally and 0.955 externally. While it ranks third in cross‑modal retrieval, Lumen’s low‑parameter training yields competitive results at both patch and slide levels.
arXiv Computer Vision
Sep 4
Solving the Needle-in-a-Haystack Problem in Mammography Vision-Language Model with Differentiable Subset Sampling
The paper introduces TopKSigLIP, a vision‑language model tailored for mammography that tackles two key challenges: high‑resolution imaging and homogeneous radiology reports. It replaces standard CLIP training with a TopK‑Patch module that selects sparse high‑resolution patches likely to contain lesions, and a Sup‑sigmoid loss that uses soft labels from structured data instead of contrastive loss. TopKSigLIP outperforms existing open‑source mammography and general medical VLMs on zero‑shot tasks such as density assessment, BI‑RADS classification, finding subtyping, and cancer prediction, while also providing better lesion localization than Grad‑CAM.
arXiv AI
Jul 13
TCLA: Training-Free Class-wise Logit Adaptation for Medical Vision-Language Models
arXiv:2607. 09562v1 Announce Type: cross Abstract: Medical Vision-Language Models (VLMs) exhibit strong zero-shot performance, yet their effectiveness still declines on out-of-distribution (OOD) data due to domain shifts and class bias inherited from large-scale pretraining.
arXiv AI
Sep 3
Federated LoRA Adaptation of BiomedCLIP Across Four International Chest X-Ray Cohorts
The paper evaluates federated learning with Low‑Rank Adaptation (LoRA) for fine‑tuning the BiomedCLIP vision‑language model on chest X‑ray classification across four international cohorts. Federated LoRA improves shared‑class AUC from 0.687 to 0.802, outperforming isolated single‑cohort training and approaching a centralized reference. The study shows that SVD‑based product‑space aggregation (FlexLoRA) is crucial for performance, while FedProx offers no advantage over FedAvg in this setting.
arXiv AI
Jul 14
Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
arXiv:2607. 10358v1 Announce Type: cross Abstract: Foundation models are increasingly used as image feature extractors for mammography, but their robustness under external domain shift remains unclear.
arXiv AI
Jun 4
Geometry-Aware Distillation for Prompt Tuning Biomedical Vision-Language Models
arXiv:2606. 04922v1 Announce Type: cross Abstract: Current prompt-based and adapter-based tuning of vision-language models (VLMs) is attractive for medical imaging, where clinical data sensitivity favors frozen backbones and annotations are limited.