arXiv Computer Vision

Can Multimodal Decision Models Classify Breast Ultrasound Images? Benchmarking CLEF and CLEF-Flash Across Prompts and Datasets

The paper evaluates multimodal decision models, specifically CLEF and CLEF‑Flash, on 2,818 breast ultrasound images from three datasets (Dataset B, BUSI, BUS‑BRA). Using three different prompts—minimal labels, explicit classification, and sonographic criteria—the study reports accuracy, macro F1, balanced accuracy, precision, throughput, and latency. CLEF‑Flash with a clinical prompt achieves the highest accuracy across all datasets, while CLEF with a task‑guided prompt yields the best macro F1 and balanced accuracy on BUSI; however, all configurations underperform a majority‑class predictor on BUS‑BRA, highlighting frequent benign‑to‑malignant misclassifications.

arXiv Computer Vision
Sep 18

Performance of Machine Learning Classification in Sonomammogram Images using BI-RADS

This study evaluates the classification accuracy of six modern deep‑learning architectures—VGG19, ResNet50, GoogleNet, ConvNeXt, EfficientNet, and Vision Transformers—on breast ultrasound images categorized by BI‑RADS. Using 2,945 training images and 936 validation images from 1,540 patients, the models were tested in full fine‑tuning, linear evaluation, and training‑from‑scratch settings. The best performance was achieved with full fine‑tuning, yielding 76.39 % accuracy and a 67.94 % F1 score.

By Malitha Gunawardhana, Norbert Zolek
arXiv Computer Vision
Oct 12

Multimodal Fusion for Fine-Grained Classification of Breast Fibroadenoma and Phyllodes Tumors

The paper introduces the FAPT-M Dataset, a pathology‑confirmed multimodal collection of 910 breast patients that includes ultrasound images, structured clinical attributes, and diagnostic descriptions. It proposes a clinically guided multimodal framework that fuses DenseNet visual encoding, CLIP‑inspired text encoding, and lightweight clinical encoding, enhanced by clinical‑conditioned adaptive modulation, cross‑modal Transformer fusion, and dual‑path representation learning. The method achieves 77.64% accuracy, 73.38% F1‑score, and 89.74% AUC, outperforming CNN, Transformer, and vision‑language baselines, and demonstrates the value of three‑modality fusion through ablation and class‑balanced studies.

By Chuxi Nan, Di Wu, Hongming Guo, Ning Cao, Xiaohui Zhu, Zhaoting Shi, Jiawei Li
Hugging Face Trending Papers
Jun 3

BreastGPT: A Multimodal Large Language Model for the Full Spectrum of Breast Cancer Clinical Routine

Breast cancer remains a leading cause of cancer-related mortality among women. Its clinical management requires multimodal reasoning across a clinical workflow that spans \textit{screening}, \textit{diagnosis} and \textit{treatment planning}, where each stage involves distinct imaging modalities, task objectives, and reasoning patterns.

arXiv AI
Jul 15

Calibrated Selective Prediction Using Deep Ensembles for ROI-Based Thyroid Nodule Ultrasound Classification Under Dataset Shift: A Retrospective Evaluation

arXiv:2607. 12075v1 Announce Type: cross Abstract: Background: Deep learning models can classify thyroid nodules on ultrasound, but reliable clinical decision support also requires calibrated probabilities, uncertainty estimation, and selective referral, particularly under dataset shift.

By Md. Sadibul Hasan Sadib, Md. Mohayminul Mukit, Rahmatul Kabir Rasel Sarker, Tahmid Alam Tamim, Md. Monir Hossain Shimul
arXiv Machine Learning
Oct 3

OmniMed-Jev: Calibrating LVLM Confidence for Trustworthy Medical Multimodal Decisions via System One

OmniMed-Jev is a new medical multimodal model that represents each decision as a Choice, Noul, or Score over a runtime-supplied candidate set, returning a full probability distribution for each decision. By unifying diverse imaging modalities and prediction tasks into a single candidate-conditioned probability model, it makes heterogeneous outputs comparable probabilities rather than task-specific strings. In controlled comparisons against a generative baseline, OmniMed-Jev’s reported probabilities align more closely with observed correctness, reducing calibration error by up to an order of magnitude and reliability error by up to two, while maintaining comparable point-prediction performance.

By Luyao Tang, Cheng Chen
arXiv AI
Jul 29

Evaluating Multi-Turn Multimodal Diagnostic Reasoning on Challenging Real-World Clinical Cases

arXiv:2607. 25933v1 Announce Type: cross Abstract: Clinical diagnostic evaluation should not only assess whether models can provide correct diagnoses, but also reflect the realities of clinical practice, including progressive disclosure of multimodal information, dynamic updating of diagnostic hypotheses, and continuous refinement of clinical reasoning.

By Rui Yang, Weihao Xuan, Yi Lin, Zhuhan Bao, Jonathan Chong Kai Liew, Matthew Yu Heng Wong, Nicol\'as Lescano, Nikita R. Paripati, Emily Ling-Lin Pai, Jiarui Liu, Heli Qi, Heng-Jui Chang, Benny Kai Guo Loo, Huitao Li, Kunyu Yu, Yufan Wang, Chuan Hong, Shijian Lu, Douglas Teodoro, Naoto Yokoya, Ross Koppel, Mona Diab, Hua Xu, David W. Bates, Nan Liu, Yifan Peng
arXiv AI
Jun 30

IMCBench: A benchmark for multimodal LLMs in Image-grounded Medical Conversations

arXiv:2606. 28556v1 Announce Type: new Abstract: Recent advances in large language models and vision-language models have enabled reasoning over multimodal data, offering opportunities for clinical applications such as decision support and triaging.

By Maria Xenochristou, Ashutosh Joshi, Korosh Vatanparvar, Mohammad Abuzar Hashemi, Prasad Kasu, Deepak Bansal, Anchal Nema, Nivedita Wadhwa, Prashams S Jain, Rebecca Abraham, Will Kimbrough, Dilek Hakkani-Tur, Wilko Schulz-Mahlendorf
arXiv Machine Learning
4d ago

Multimodal Medical Image Parsing with Task-Specific Low-Rank Adapters

The paper presents a method for adapting a single vision‑language model, Qwen3.5‑9B, to medical image parsing tasks by using task‑specific low‑rank adapters (LoRA) selected from the input question. The system employs a frozen 4‑bit backbone and customizes adapter ranks, prompts, and class rebalancing across six task families and seven imaging modalities, achieving strong performance on the FLARE 2026 Task 3 dataset and a hidden‑validation set. The authors provide code and demonstrate that task‑specific adapter configurations and image budgets improve performance across diverse medical imaging tasks.

By Taha Sabir, Alberto Montero Molina, Thinh Quang Ly, Veronika Dmitrenko, Mohammad Yaqub, Tausifa Jan Saleem
arXiv AI
Aug 5

CorePath: A Breast-Specialized Pathology Foundation Model for Core Needle Biopsy Diagnosis and Risk-Controlled Report Generation

arXiv:2608. 03079v1 Announce Type: cross Abstract: Breast core needle biopsy (CNB) is central to breast cancer diagnosis yet remains challenging because limited tissue sampling, lesion heterogeneity, and subtle morphologic overlap can obscure subtype distinctions.

By Ting Yin, Danning Li, Chen Shu, Xiaoxia Yao, Boyu Fu, Yujing Chang, Tianyu Shi, Mengna Feng, Jie Chen, Jing Fu, Xiuli Xiao, Tianlin Li, Mumin Shao, Jiaxin Bi, Wenchuan Zhang, Xiaoyan Wu, Xiao Han, Zhang Zhang, Yuhao Yi, Hong Bu
arXiv Computer Vision
Oct 12

Thyroid Nodule Segmentation in Ultrasound Images with Conformal Uncertainty Estimation

The paper presents a thyroid nodule segmentation framework for ultrasound images that combines DeepLabV3+ with EfficientNet‑B5, channel‑attention modules (Squeeze‑and‑Excitation, CBAM, GCA), and a post‑hoc conformal prediction layer to provide pixel‑level uncertainty estimates. Experiments on the TN3K benchmark show that the CBAM‑augmented model with Dice‑BCE loss outperforms existing methods. The approach aims to improve the first stage of a CAD pipeline for malignancy classification and metastatic risk assessment.

By Francesco Teodori, Pierpaolo Brutti, Umberto Ferraro Petrillo