arXiv Computer Vision By Tran Cao Minh, Hai Tran, Ha Minh Tan

Can Multimodal Decision Models Classify Breast Ultrasound Images? Benchmarking CLEF and CLEF-Flash Across Prompts and Datasets

Read the original on arXiv Computer Vision →

The paper evaluates multimodal decision models, specifically CLEF and CLEF‑Flash, on 2,818 breast ultrasound images from three datasets (Dataset B, BUSI, BUS‑BRA). Using three different prompts—minimal labels, explicit classification, and sonographic criteria—the study reports accuracy, macro F1, balanced accuracy, precision, throughput, and latency. CLEF‑Flash with a clinical prompt achieves the highest accuracy across all datasets, while CLEF with a task‑guided prompt yields the best macro F1 and balanced accuracy on BUSI; however, all configurations underperform a majority‑class predictor on BUS‑BRA, highlighting frequent benign‑to‑malignant misclassifications.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv Computer Vision
Sep 18

Performance of Machine Learning Classification in Sonomammogram Images using BI-RADS

This study evaluates the classification accuracy of six modern deep‑learning architectures—VGG19, ResNet50, GoogleNet, ConvNeXt, EfficientNet, and Vision Transformers—on breast ultrasound images categorized by BI‑RADS. Using 2,945 training images and 936 validation images from 1,540 patients, the models were tested in full fine‑tuning, linear evaluation, and training‑from‑scratch settings. The best performance was achieved with full fine‑tuning, yielding 76.39 % accuracy and a 67.94 % F1 score.

By Malitha Gunawardhana, Norbert Zolek
arXiv Computer Vision
Oct 12

Multimodal Fusion for Fine-Grained Classification of Breast Fibroadenoma and Phyllodes Tumors

The paper introduces the FAPT-M Dataset, a pathology‑confirmed multimodal collection of 910 breast patients that includes ultrasound images, structured clinical attributes, and diagnostic descriptions. It proposes a clinically guided multimodal framework that fuses DenseNet visual encoding, CLIP‑inspired text encoding, and lightweight clinical encoding, enhanced by clinical‑conditioned adaptive modulation, cross‑modal Transformer fusion, and dual‑path representation learning. The method achieves 77.64% accuracy, 73.38% F1‑score, and 89.74% AUC, outperforming CNN, Transformer, and vision‑language baselines, and demonstrates the value of three‑modality fusion through ablation and class‑balanced studies.

By Chuxi Nan, Di Wu, Hongming Guo, Ning Cao, Xiaohui Zhu, Zhaoting Shi, Jiawei Li
Hugging Face Trending Papers
Jun 3

BreastGPT: A Multimodal Large Language Model for the Full Spectrum of Breast Cancer Clinical Routine

Breast cancer remains a leading cause of cancer-related mortality among women. Its clinical management requires multimodal reasoning across a clinical workflow that spans \textit{screening}, \textit{diagnosis} and \textit{treatment planning}, where each stage involves distinct imaging modalities, task objectives, and reasoning patterns.

arXiv AI
Jul 15

Calibrated Selective Prediction Using Deep Ensembles for ROI-Based Thyroid Nodule Ultrasound Classification Under Dataset Shift: A Retrospective Evaluation

arXiv:2607. 12075v1 Announce Type: cross Abstract: Background: Deep learning models can classify thyroid nodules on ultrasound, but reliable clinical decision support also requires calibrated probabilities, uncertainty estimation, and selective referral, particularly under dataset shift.

By Md. Sadibul Hasan Sadib, Md. Mohayminul Mukit, Rahmatul Kabir Rasel Sarker, Tahmid Alam Tamim, Md. Monir Hossain Shimul
arXiv Machine Learning
Oct 3

OmniMed-Jev: Calibrating LVLM Confidence for Trustworthy Medical Multimodal Decisions via System One

OmniMed-Jev is a new medical multimodal model that represents each decision as a Choice, Noul, or Score over a runtime-supplied candidate set, returning a full probability distribution for each decision. By unifying diverse imaging modalities and prediction tasks into a single candidate-conditioned probability model, it makes heterogeneous outputs comparable probabilities rather than task-specific strings. In controlled comparisons against a generative baseline, OmniMed-Jev’s reported probabilities align more closely with observed correctness, reducing calibration error by up to an order of magnitude and reliability error by up to two, while maintaining comparable point-prediction performance.

By Luyao Tang, Cheng Chen