arXiv AI

CRS-Bench: A Reference-Relative Reliability Benchmark for Medical Image Encoders

arXiv AI
Sep 25

Med-AR: Autoregressive Vision-Language Pretraining for Long-Tailed Chest X-Ray Classification and Uncertainty-Aware Evaluation

Med-AR introduces two autoregressive vision‑language models, Med‑AR‑8B and Med‑AR‑2B, pretrained on structured radiology reports, abnormality‑focused text, and region annotations to address long‑tailed chest X‑ray classification. The models outperform existing contrastive, self‑supervised, and supervised encoders—including Med‑CLIP, CheXFound, EVA‑Base, ARK, and BioViL‑T—across PadChest, MIMIC‑CXR, and CheXpert, achieving higher mean AUROC and AUPRC for head, medium, and tail findings and lower excess area under the risk‑coverage curve. Med‑AR also demonstrates improved selective‑prediction performance, with Med‑AR‑8B raising tail‑label mean AUPRC on MIMIC‑CXR from 0.1033 to 0.1441 and Med‑AR‑2B delivering the strongest discrimination on PadChest.

By Janhavi Prabhu, Sahil, Akshay V, Shivam Shukla, Manoj Tadepalli, Preetham Putha
arXiv AI
Sep 15

Adaptive Conformal Redistribution for Inter-class Transitional Uncertainty in Medical Image Classification

The paper introduces AdaConRed, a label‑free post‑conformal decision rule that transforms ambiguous conformal prediction sets into single class assignments for medical image classification. It employs a five‑stage pipeline—vision‑language generative augmentation, a frozen DermFoundation encoder, a lightweight MLP classifier, an entropy‑modulated margin‑aware nonconformity score, and a reassignment step for transitional samples—to improve accuracy on OSCC and ISIC benchmarks. Results show notable gains in malignant class accuracy while maintaining overall performance.

By Saibal Ghosh, Samarup Bhattacharya, Sanjoy Kumar Saha, Umapada Pal, Tapabrata Chakraborti
arXiv AI
Sep 7

Cross-modal triage network: a multimodal deep learning framework for severity-based triage and visual explainability in chest radiographs

The paper introduces the Cross‑Modal Triage Network (CMTN), a multimodal deep‑learning model that fuses a Swin Transformer V2 visual encoder with a PubMedBERT text encoder to perform severity‑based triage, pathology detection, and generate visual explanations for chest radiographs. Trained on 34,639 image‑text pairs from MIMIC‑CXR‑JPG, the CMTN achieves high ordinal agreement with reference labels (QWK = 0.9341) and excellent pathology detection (macro‑AUROC = 0.9970) while operating with 34 ms latency. However, a blinded clinical audit revealed low agreement with expert radiologists (QWK = 0.1399) and only modest spatial‑semantic concordance in heatmaps, underscoring the gap between algorithmic performance and clinical judgment.

By Zinah Ghulam, Richa Mittal, Eranga Ukwatta
arXiv Computer Vision
2d ago

A Decision-Aware Evaluation Protocol for Uncertainty in Medical Image Segmentation

The paper introduces a decision‑aware evaluation protocol for medical image segmentation uncertainty that separates uncertainty estimation from deferral policy and assesses their joint performance under fixed review budgets. It reports metrics such as fixed‑budget error reduction, risk‑coverage behavior, region‑level review proxies, structural failure modes, and sensitivity to review assumptions. Experiments on retinal vessel, skin lesion, and polyp segmentation with various backbones and uncertainty estimators show that estimator rankings vary with review unit, uncertainty target, and cost model, and that boundary and thin‑structure failures are partly due to annotation ambiguity.

By Saket Maganti, Shreyank N Gowda
arXiv Machine Learning
Jul 30

Rethinking Clinical Relevance in Chest X-ray Machine Learning: How Evaluation References Define Performance

arXiv:2607. 26333v1 Announce Type: cross Abstract: Chest X-ray (CXR) machine learning relies heavily on automated evaluation using reference standards that aim to approximate clinical judgment.

By Panagiotis Fytas, Ian Selby, Clemens Karner, Judith Babar, Simon Baker, Jake Beckford, Timothy J. Sadler, Shahab Shahipasand, Arthikkaa Thavakumar, John Li Chen, Alex Sawer, Michael Roberts, Jonathan Weir-McCall, J. H. F. Rudd, Carola-Bibiane Sch\"onlieb, Anna Korhonen, Anna Breger
arXiv Computer Vision
Sep 17

Lumen: Parameter-Efficient Alignment of Pretrained Vision and Language Encoders for Zero-Shot Computational Pathology

Lumen is a pathology vision‑language model that aligns frozen unimodal foundation models (Virchow2 and BioMedBERT) using rank‑4 adapters and projection heads, training only 0.40% of the total parameters on the QUILT‑1M corpus. It achieves the highest mean chance‑corrected balanced accuracy (0.546) across nine zero‑shot patch benchmarks and demonstrates strong performance on lymph‑node metastasis detection, with AUROC scores of 0.964 internally and 0.955 externally. While it ranks third in cross‑modal retrieval, Lumen’s low‑parameter training yields competitive results at both patch and slide levels.

By Kiarash Tajbakhsh, Abdelrahman Faqieh, Michael Jopiti, Javier Garcia-Baroja, Philipp Zens, Branislav Zagrapan, Yuri Tolkach, Martin D. Berger, Aurel Perren, Bastian Dislich, Inti Zlobec, Amjad Khan
arXiv Computer Vision
Aug 28

MVC-Bench: Benchmarking Calibration of Medical Vision-Language Models

MVC-Bench is a new benchmark designed to evaluate the calibration of vision‑language models (VLMs) and medical VLMs (Medical‑VLMs) for medical image classification. It tests calibration across robustness to modality, backbone, and domain shift; effectiveness of calibration strategies and prompt‑tuning methods; and stability under prompt‑template and random‑seed variations. The benchmark includes eight backbones, three medical modalities (fundus imaging, histopathology, chest X‑ray), and compares post‑hoc, train‑time, and zero‑shot calibration approaches, reporting accuracy, Expected Calibration Error (ECE), Maximum Calibration Error (MCE), and Adaptive Calibration Error (ACE) over 1,638 experiments, while also proposing a Multi‑Class Margin (MCM) regularization technique that improves ECE in most settings.

By Ashshak Sharifdeen, Shihab Aaqil Ahamed, Ufaq Khan, Muhammad Akhtar Munir Sujair Ibrahim, Mohamed Rafeek Mareer Ahamed, Yutong Xie, Imran Razzak, Muhammad Haris Khan
arXiv Computer Vision
Sep 7

Cross-dataset transportability of pediatric chest X-ray deep learning across three countries: discrimination, calibration, operating-point failure, and limited-label recovery

The study evaluates a deep‑learning model for pediatric pneumonia detection across chest X‑ray datasets from three countries, assessing discrimination, calibration, operating‑point transport, shortcut signals, and limited‑label recovery. Using a frozen DenseNet121 ensemble trained on Guangzhou data, the model achieved high internal AUROC (0.976) but performance dropped when applied zero‑shot to Bangladesh (AUROC 0.798) and Vietnam (AUROC 0.742). Limited‑label adaptation with Platt recalibration restored sensitivity but introduced significant specificity variability, highlighting the need to evaluate multiple performance dimensions in cross‑dataset transport studies.

By Nazim-E-Alam
arXiv Machine Learning
3d ago

Multimodal Medical Image Parsing with Task-Specific Low-Rank Adapters

The paper presents a method for adapting a single vision‑language model, Qwen3.5‑9B, to medical image parsing tasks by using task‑specific low‑rank adapters (LoRA) selected from the input question. The system employs a frozen 4‑bit backbone and customizes adapter ranks, prompts, and class rebalancing across six task families and seven imaging modalities, achieving strong performance on the FLARE 2026 Task 3 dataset and a hidden‑validation set. The authors provide code and demonstrate that task‑specific adapter configurations and image budgets improve performance across diverse medical imaging tasks.

By Taha Sabir, Alberto Montero Molina, Thinh Quang Ly, Veronika Dmitrenko, Mohammad Yaqub, Tausifa Jan Saleem
arXiv AI
Jul 23

Self-supervision drives representational convergence in medical foundation models more than clinical supervision

arXiv:2607. 20274v1 Announce Type: cross Abstract: Medical image encoders from different groups are increasingly treated as interchangeable, on the assumption that scale and clinical supervision concentrate their representations onto a shared structure.

By Soroosh Tayebi Arasteh, Sebastian Ziegelmayer, Mahshad Lotfinia, Lisa Adams, Sven Nebelung, Jakob Nikolas Kather, Daniel Truhn
arXiv Computer Vision
Sep 30

HERO: Histology Encoder for Robust Representation in Oncology

HERO (Histology Encoder for Robust Representation in Oncology) is a ViT‑G/14 pathology foundation model trained with DINO and iBOT objectives and refined using high‑resolution Gram anchoring on a 500‑million‑tile corpus from about 575,000 clinical whole‑slide images. It demonstrates superior robustness to center, scanner, and stain variation compared to other state‑of‑the‑art foundation models, while maintaining competitive performance on tile‑level classification, segmentation, and gene‑expression prediction. Across 39 slide‑level clinical tasks, HERO ranks first on average and achieves the best average rank across six benchmark frameworks under an equal‑weighted analysis.

By Zhi Li (Caris Life Sciences, Irving, TX, United States), Eghbal Amidi (Caris Life Sciences, Irving, TX, United States), Yating Cheng (Caris Life Sciences, Irving, TX, United States), Tyson Dawson (Caris Life Sciences, Irving, TX, United States), Gorkem Can Ates (Caris Life Sciences, Irving, TX, United States), Shuzhen Kuang (Caris Life Sciences, Irving, TX, United States), Norsang Lama (Caris Life Sciences, Irving, TX, United States), Md Ashequr Rahman (Caris Life Sciences, Irving, TX, United States), Zhiying Lu (Caris Life Sciences, Irving, TX, United States), Elisabeth K. Kong (Caris Life Sciences, Irving, TX, United States), Milan Radovich (Caris Life Sciences, Irving, TX, United States), David Spetzler (Caris Life Sciences, Irving, TX, United States), Matthew Oberley (Caris Life Sciences, Irving, TX, United States), George W. Sledge (Caris Life Sciences, Irving, TX, United States), Ming Chen (Caris Life Sciences, Irving, TX, United States)
arXiv Computer Vision
Aug 27

Can You Trust Frozen Hematology Foundation Models under Acquisition Shift?

The study evaluates 15 frozen hematology foundation-model embeddings across four single‑cell acquisition domains, finding that while in‑domain accuracy is near‑saturated (macro‑F1 0.98–0.997), cross‑dataset performance drops dramatically (34–72%) and model rankings shift. Probe‑dependent rank transfer is observed, with 1‑NN retrieval more stable than linear heads, yet neither reliably predicts target robustness. Calibration deteriorates off‑domain (ECE rises from 0.004 to 0.35), and exposure to internal cohorts confounds shift analysis; a training‑free pseudo‑label‑balanced feature normalization (CBR) modestly improves target‑prior robustness and calibration. whyItMatters":"The findings highlight that frozen hematology foundation models, though accurate in‑domain, may fail under realistic scanner, site, and class‑prior shifts, underscoring the need for comprehensive audits of accuracy, calibration, exposure, and robustness before clinical deployment."

By Jai Kumar Sharma, Peeyush Tapadiya