arXiv Computer Vision

SCCL-DT: Semantic-Conflict-Aware Contrastive Learning with Dual-Rule Teacher Distillation for Zero-Shot Chest X-Ray Diagnosis

The paper introduces SCCL-DT, a method that improves zero‑shot chest X‑ray diagnosis by addressing semantic conflicts in contrastive vision‑language pre‑training. It uses adaptive thresholds derived from exponential‑moving‑average statistics to hard‑mask conflicting pairs, and combines two teachers—an inclusive and a consensus rule—through distillation into a single student model. On five public chest X‑ray benchmarks, SCCL‑DT achieves a mean zero‑shot AUC of 0.850, outperforming the best baseline at 0.837.

arXiv AI
Oct 27

Meta-CLIP: Metadata-Conditioned Contrastive Learning for Zero-Shot Disease Inference

Meta-CLIP is a metadata‑conditioned contrastive learning framework that incorporates DICOM acquisition information to reweight cross‑modal negatives during medical vision‑language pretraining. By defining a bounded pairwise margin over numerical and categorical DICOM fields, the method emphasizes distinctions among studies with similar imaging conditions while keeping the kernel fixed during optimization. Evaluations on chest radiography and volumetric CT show that Meta‑CLIP improves zero‑shot classification performance, achieving higher macro‑AUROC scores than baseline CLIP models on multiple benchmarks.

By Haozhe Luo, Nirosh Sivanesan, Shelley Zixin Shu, Francesco Nitti, Mauricio Reyes
arXiv Machine Learning
Jul 17

Multimodal Semantic-Aware Contrastive Learning For False Negative Mitigation in 3D Medical Imaging

arXiv:2607. 14995v1 Announce Type: new Abstract: Multimodal Contrastive Learning (CL) has shown significant performance in aligning representations across various data modalities and improving downstream tasks, especially in healthcare.

By Sara Ketabi, Matthias W. Wagner, Cynthia Hawkins, Uri Tabori, Birgit Betina Ertl-Wagner, Farzad Khalvati
arXiv Machine Learning
Aug 4

RadPRISM: Schema-stratified radiology-report supervision for concept-disentangled image representations and visual grounding

arXiv:2608. 00147v1 Announce Type: cross Abstract: Vision-language pretraining learns rich medical image representations from radiology reports, but previous model variants commonly operate within a single shared embedding space, so concept-level structure and interpretability must be recovered post hoc, limiting model transparency and, hence, clinical utility.

By Fabian Drexel, Marlene Fritzsche, Era Stambollxhiu, Miriam Kumpf, Lena Schmitzer, Lea Schumann, Jannik Kahmann, Friedrich Puttkammer, Johannes Moll, Jannik L\"ubberstedt, Zeineb Ben Chaaben, Anirudh Narayanan, Cosmin I. Bercea, Sebastian Ziegelmayer, Marcus R. Makowski, Daniel Rueckert, Lisa C. Adams, Keno K. Bressem
arXiv Computer Vision
Oct 9

Similarity-as-Evidence: Calibrating Overconfident VLMs for Interpretable and Label-Efficient Medical Active Learning

The paper introduces Similarity-as-Evidence (SaE), a framework that calibrates overconfident Vision‑Language Models (VLMs) for medical active learning by reinterpreting text‑image similarity vectors as evidence and modeling them with a Dirichlet distribution. SaE’s Similarity Evidence Head (SEH) captures uncertainty through vacuity and dissonance, enabling a dual‑factor acquisition strategy that prioritizes high‑vacuity samples early and high‑dissonance samples later. Experiments on ten public medical imaging datasets demonstrate state‑of‑the‑art macro‑averaged accuracy (82.57%) and improved calibration (NLL = 0.425) under a 20% label budget.

By Zhuofan Xie, Zishan Lin, Jinliang Lin, Jie Qi, Shaohua Hong, Shuo Li
arXiv Computer Vision
Sep 3

AlphaRAD: Grounded Zero-Shot Classification in Chest Radiology via $\alpha$-Corrected Binary Cross Entropy and Factorized Latent Supervision

AlphaRAD introduces a grounded zero‑shot classification framework for chest radiology that leverages structured medical concepts extracted from reports and a novel α‑Corrected Binary Cross‑Entropy loss to reduce in‑batch noise. It also presents FLaS, a lightweight cross‑modal fusion module that factorizes VLPM representations into independent subspaces, improving spatial grounding without adding parameters. The method achieves state‑of‑the‑art performance on 16 classification benchmarks and sets new records on several grounding, phrase‑grounding, and segmentation datasets.

By Jianzhong You, Yuan Gao, Chris McIntosh
arXiv AI
Jul 8

Harrison.Rad 1.5 Technical Report: A radiology foundation model that can draft reports from images, priors and clinical context

arXiv:2607. 05880v1 Announce Type: cross Abstract: Imaging demand is growing faster than the radiology workforce can expand, and reporting backlogs cannot be resolved through training and recruitment alone.

By Suneeta Mall, Vladimir Nekrasov, Ashnil Kumar, Sajith Karunasena, Aiden Nibali, Alix Bird, Mateo Diaz Shine, Jarrel Seah
arXiv AI
Sep 11

OmniMed-FL: A Robust Multimodal Federated Learning Framework for Clinical Diagnosis

OmniMed‑FL is a multimodal federated learning framework that fuses chest radiographs and synthetic patient notes to classify five clinical conditions. The study benchmarks eight fusion strategies, three initializations, and four missing‑text imputation rules across 3–20 hospital clients under non‑IID Dirichlet partitioning, showing that federated approaches (FedAvg, FedProx, SCAFFOLD‑AdamW) outperform local‑only training. Multimodal fusion consistently improves performance, achieving macro‑F1 scores up to 0.956 on the synthetic corpus and 0.906 on the radiograph corpus.

By Ayush Debnath, Ruelia Saha, Sudip Misra
Hugging Face Trending Papers
Jun 24

Disease-Centric Vision-Language Pretraining with Hybrid Visual Encoding for 3D Computed Tomography

Vision-language pre-training (VLP) holds great promise for general-purpose medical AI by leveraging radiology reports as rich textual supervision, yet existing methods struggle with 3D CT imaging due to inefficient visual backbones and coarse semantic alignment. To address these issues, we propose a tailored VLP framework featuring three key components: (1) a CNN-ViT hybrid encoder that replaces ViT's patch embedding with a 3D CNN backbone to efficiently capture local anatomical details while preserving global attention and compatibility with pre-trained cross-modal priors; (2) a disease-level contrastive learning mechanism using learnable query tokens to dynamically extract disease-specific semantics from full reports and align them with corresponding visual features, thereby disentangling distinct diseases within the same anatomical region; and (3) a diagnosis-aware prompt strategy that employs real clinical phrases and aggregated disease prototypes to bridge the pre-training-inference gap and enhance zero-shot diagnostic reliability.

arXiv AI
Sep 25

Med-AR: Autoregressive Vision-Language Pretraining for Long-Tailed Chest X-Ray Classification and Uncertainty-Aware Evaluation

Med-AR introduces two autoregressive vision‑language models, Med‑AR‑8B and Med‑AR‑2B, pretrained on structured radiology reports, abnormality‑focused text, and region annotations to address long‑tailed chest X‑ray classification. The models outperform existing contrastive, self‑supervised, and supervised encoders—including Med‑CLIP, CheXFound, EVA‑Base, ARK, and BioViL‑T—across PadChest, MIMIC‑CXR, and CheXpert, achieving higher mean AUROC and AUPRC for head, medium, and tail findings and lower excess area under the risk‑coverage curve. Med‑AR also demonstrates improved selective‑prediction performance, with Med‑AR‑8B raising tail‑label mean AUPRC on MIMIC‑CXR from 0.1033 to 0.1441 and Med‑AR‑2B delivering the strongest discrimination on PadChest.

By Janhavi Prabhu, Sahil, Akshay V, Shivam Shukla, Manoj Tadepalli, Preetham Putha