The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Multimodal based approaches often outperform single modality approaches in downstream tasks as the different modalities provide complementary information, yet acquiring paired clinical data remains a significant challenge in real world scenarios. While cross-modal knowledge distillation addresses this, existing methods often struggle with large modality gaps and the propagation of noise from uncertain source-domain predictions.
arXiv:2608. 13690v1 Announce Type: cross Abstract: Medical image segmentation is still largely treated as a vision-only problem, although clinical interpretation often relies on textual knowledge of anatomy, location, appearance, and surrounding context.
The paper introduces an unsupervised domain adaptation framework that aligns redundancy-reducing features to enable accurate 3D segmentation of cone-beam CT (CBCT) without target-domain annotations or inference-time adaptation. The method is architecture-agnostic, working with both CNN-based and ViT-based foundation models, and is evaluated on two liver segmentation benchmarks for interventional vascular procedures and radiation therapy. Results show that even large pretrained segmentation networks need explicit feature-space bridging to generalize across diagnostic CT and CBCT, and the proposed approach consistently outperforms existing pretrained foundation models and UDA strategies.
SEG-SAM is a unified medical image segmentation model that builds on the Segment Anything Model (SAM) by integrating semantic medical knowledge. It introduces a semantic‑aware decoder separate from SAM’s original decoder to handle both semantic segmentation of prompted objects and classification of unprompted objects. The model also incorporates key medical category characteristics from large language models via a text‑to‑vision semantic module and uses a cross‑mask spatial alignment strategy to improve overlap between predictions, achieving superior performance over existing SAM‑based and task‑specific methods.
arXiv:2608. 05683v1 Announce Type: cross Abstract: Cross-modal alignment of visual and textual representations is fundamental to multimodal medical image understanding, yet remains hindered by uncertainty in both modalities under real-world clinical conditions.
arXiv:2608. 19825v1 Announce Type: cross Abstract: Medical image captioning is a technique that accelerates early-stage diagnostic workflows and enhances the interpretability of medical diagnostic AI systems.