arXiv Computer Vision

Solving the Needle-in-a-Haystack Problem in Mammography Vision-Language Model with Differentiable Subset Sampling

The paper introduces TopKSigLIP, a vision‑language model tailored for mammography that tackles two key challenges: high‑resolution imaging and homogeneous radiology reports. It replaces standard CLIP training with a TopK‑Patch module that selects sparse high‑resolution patches likely to contain lesions, and a Sup‑sigmoid loss that uses soft labels from structured data instead of contrastive loss. TopKSigLIP outperforms existing open‑source mammography and general medical VLMs on zero‑shot tasks such as density assessment, BI‑RADS classification, finding subtyping, and cancer prediction, while also providing better lesion localization than Grad‑CAM.

arXiv Computer Vision

Oct 9

Healthy Counterfactual Generation via Diffusion Inpainting for Mammography Classification

The paper introduces a counterfactual data augmentation method that generates healthy mammograms by inpainting lesions with realistic normal tissue using a Denoising Diffusion Probabilistic Model and a RePaint-based sampling strategy. The generated healthy counterfactuals preserve patient-specific anatomy and match the quality of the original dataset, as confirmed by similarity metrics and radiologist assessment. When applied to four different classifier architectures on the VinDr-Mammo dataset, the augmentation improves sensitivity, especially at 80% fixed specificity, thereby enhancing the reliability of computer-aided breast cancer screening systems.

By In\^es Cruchinho Garcia, Mariana Mour\~ao, Francisco Maria Calisto, Carlos Santiago, Jacinto Nascimento

llms diffusion computer-vision multimodal

arXiv AI

Sep 1

ActiveAugment: Online Active Learning for Augmentation Selection in Deep Learning

arXiv:2608.28923v1 Announce Type: cross Abstract: Data augmentation is a cornerstone of deep learning pipelines, yet existing strategies treat it as a static, model-agnostic preprocessing step, eithe...

By Noah Videcrantz, Mostafa Mehdipour Ghazi

llms fine-tuning benchmarks

arXiv AI

Jul 14

Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift

arXiv:2607. 10358v1 Announce Type: cross Abstract: Foundation models are increasingly used as image feature extractors for mammography, but their robustness under external domain shift remains unclear.

By Giang Nguyen, Raghav Mehta, Emma A. M. Stanley, Tian Xia, Thi Hao Nguyen, Hieu Pham, Ben Glocker

llms multimodal benchmarks

arXiv Machine Learning

Jun 3

GLINT: Sparsely Gated Vision-Language Alignment for Fine-Grained Radiology Representations

arXiv:2606. 03180v1 Announce Type: cross Abstract: Vision-language models (VLMs) for radiology have emerged as a scalable paradigm by leveraging image-report pairs naturally produced in clinical workflows.

By Jonggwon Park, Seongeun Lee, Junhyun Park, Hannah Yun, Hyunwoong Kim, Sohyun Jeong, Hyewon Kang, Byungmu Yoon, Kyoyun Choi

llms rag computer-vision efficiency multimodal safety

arXiv Machine Learning

Jul 14

BiLoG-Net: A Bi-Context Location-Guided Network for Breast Mass Segmentation and Malignancy Classification in Mammography

arXiv:2607. 10188v1 Announce Type: cross Abstract: Breast cancer remains the most commonly diagnosed malignancy among women worldwide, yet accurate detection and characterization of breast masses in mammography remain challenging due to subtle intensity variations, heterogeneous tissue densities, and indistinct lesion boundaries that complicate radiological interpretation.

By Abu Fatema Mohammad Abdun Noor, Md Imam Ahasan, Md Samiul Ahasan, Kah Ong Michael Goh, S M Hasan Mahmud, Raihana Zannat

llms computer-vision benchmarks

arXiv Computer Vision

Sep 25

OncoVision: Integrating Mammography and Clinical Data through Attention-Driven Multimodal AI for Enhanced Breast Cancer Diagnosis

OncoVision is a privileged‑information training framework that learns from mammography images and clinical data during training but performs inference using only mammographic images. It employs an attention‑based encoder‑decoder to jointly segment masses, calcifications, axillary findings, and breast tissue, and predicts ten structured clinical features such as BI‑RADS. Two late‑fusion strategies (Independent and Dependent) integrate imaging, radiomic, and clinical information to improve diagnostic precision, and a retrospective multi‑reader study showed higher diagnostic confidence, reduced reading time, and segmentation accuracy comparable to or better than radiologists.

By Istiak Ahmed, Galib Ahmed, K. Shahriar Sanjid, Md. Tanzim Hossain, Md. Nishan Khan, Md. Misbah Khan, Md. Arifur Rahman, Sheikh Anisul Haque, Sharmin Akhtar Rupa, Mohammed Mejbahuddin Mia, Mahmud Hasan Mostofa Kamal, Md. Mostafa Kamal Sarker, M. Monir Uddin

computer-vision multimodal

arXiv AI

Aug 11

Modern Backbones Improve Multi-task DETR for Mammography Classification and Lesion Localization

arXiv:2608. 09801v1 Announce Type: cross Abstract: Joint exam-level prediction and candidate-region localization may improve the usefulness of AI support in mammography.

By Dinh Tan Nguyen, Quang-Hien Kha, Le-Hoang Nguyen, Minh-Toan Dinh, Xuan-Huy Nguyen, Dac Phu Ho, Cao Truong Tran, Sai Ho Ling, Lan T Ho-Pham, Liem Pham, Nguyen Quoc Khanh Le

arXiv Computer Vision

Aug 24

When Adaptation Hurts: Connecting Representational Drift to OOD Failures in MedSAM Fine-Tuning

arXiv:2608.21300v1 Announce Type: new Abstract: Foundation models for medical image segmentation, like prompt-based MedSAM, generalize well across domains and modalities, often in zero or few-shot se...

By Marko Haralovi\'c, Sounic Akkaraju, Carlo Baretta, Vasil Zapryanov, Alexia Briassouli

computer-vision fine-tuning benchmarks safety

arXiv Computer Vision

Aug 31

ARC-CT: Anatomy-Routed Contrastive Vision-Language Learning for 3D Chest CT

arXiv:2608.28455v1 Announce Type: new Abstract: Contrastive vision-language learning uses paired chest CT volumes and radiology reports to learn abnormality classifiers without manually annotated lab...

By Huseyin Umut Isik, Mehmet Alp Ozaydin, Sila Kurugol, \c{S}eyda Ertekin

llms rag multimodal safety

arXiv Computer Vision

4d ago

Beyond Sparse Captions: Aligning Slide-Level Text and Patch-Level Vision in Pathology

The paper introduces Locked Image Multiple Instance PreTraining (LIMIT), a method that aligns slide‑level pathology reports with patch‑level visual features by freezing a vision encoder and fine‑tuning a text encoder. LIMIT uses report embeddings as cross‑attention queries over bags of patch features, optimizing slide representations via InfoNCE. Applying LIMIT, the authors create CONCH‑Z, which achieves state‑of‑the‑art zero‑shot performance on 22 pathology tasks, outperforming vision‑language encoders trained on curated caption datasets and narrowing the gap with larger slide‑level foundation models.

By Daniel Shao, Dongmin Bang, Luca L. Weishaupt, Sophia J. Wagner, Nic G. Reitsam, Ming Y. Lu, Long P. Le, Richard J. Chen, Faisal Mahmood

rag fine-tuning multimodal benchmarks

arXiv Computer Vision

Aug 25

Extending the Horizon of Early Diagnosis: Lung Cancer Prediction with Vision Transformers

arXiv:2608.21571v1 Announce Type: new Abstract: Lung cancer remains a leading cause of cancer-related mortality worldwide, and early diagnosis is critical for improving survival. However, early-stage...

By Olivera Kotevska, Ian Goethert, Michael McGee, Maria Mahbub, Sean R. Wilkinson, Rowena Yip, Myvizhi Esai Selvan, Zeynep H. Gumus, Claudia Henschke, Robert J. Klein, Providencia Morales, Samuel M Aguayo, Ioana Danciu, Mayanka Chandrashekar

llms fine-tuning

arXiv AI

Aug 25

Robust Lightweight Deep Learning Models for Oral Cancer Screening

arXiv:2608.21583v1 Announce Type: new Abstract: Oral cancer is a leading cause of mortality in low-to-middle-income countries, where a shortage of specialists delays diagnosis. While point-of-care sc...

By Siddhant Bharadwaj, Aakash Shedsale, Tejashree Subramanya, Mohd. Azfar, Praveen Birur, Debnath Pal, Shankararama Sharma, Anupama Shetty, Rajesh Sundaresan

llms efficiency benchmarks safety data-engineering

Related stories