arXiv Computation and Language November 4, 2026 By Soroosh Tayebi Arasteh, Mahshad Lotfinia, Sven Nebelung, Daniel Truhn

Selective Linkage Precision for Released Medical Vision-Language Models

Read the original on arXiv Computation and Language →

The paper introduces selective linkage precision, a metric for evaluating how accurately vision‑language models can match de‑identified chest radiographs to their reports when only certain attributes (such as sex, age, projection, or acquisition year) are available. Experiments on eight public VLMs and three chest‑radiograph datasets show that filtering by these attributes dramatically improves precision—from a baseline of 1.98% to 67.68% with sex, age, and projection, and up to 99.05% when acquisition year is added—while keeping the candidate set manageable. The study demonstrates that this approach can be applied without retraining the models and may guide hospitals in deciding which metadata to release.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv AI

Jun 2

Cross-modal linkage risk in clinical vision-language models

arXiv:2606. 02276v1 Announce Type: cross Abstract: Vision-language models (VLMs) trained on paired chest radiographs and radiology reports learn a shared embedding space that can preserve instance-level image-report correspondence.

By Soroosh Tayebi Arasteh, Mahshad Lotfinia, Sven Nebelung, Daniel Truhn

llms rag fine-tuning multimodal benchmarks safety

arXiv AI

Jun 17

Vision-language models for chest radiography do not always need the image

arXiv:2606. 17710v1 Announce Type: cross Abstract: Medical vision-language models report strong chest radiograph accuracy, and this is increasingly read as evidence that they use the image.

By Mahshad Lotfinia, Sebastian Ziegelmayer, Lisa Adams, Daniel Truhn, Andreas Maier, Soroosh Tayebi Arasteh

llms multimodal benchmarks

arXiv Machine Learning

Aug 11

Auditing Medical Vision-Language Models on Chest Radiographs: Estimating Reference Agreement Across Institutions

arXiv:2608. 07550v1 Announce Type: cross Abstract: Vision-language models return structured chest-radiograph findings through interfaces exposing no confidence score, so a receiving institution cannot read off how far to trust an individual judgment.

By Pengyang Yu, Yiou Wang, Zhongping Dong, Sahraoui Dhelim, Chun-Mei Feng, M. Tahar Kechadi

llms multimodal

arXiv Computation and Language

Aug 21

Calibrated Triage, Not Autonomy: Confidence Estimation for Medical Vision-Language Models

arXiv:2606. 15910v2 Announce Type: replace Abstract: A vision-language model can answer a question about a chest radiograph or a pathology slide fluently and confidently while barely using the image, relying instead on language priors.

By Reza Khanmohammadi, Kundan Thind, Mohammad M. Ghassemi

llms agents multimodal

arXiv Machine Learning

Oct 22

From Theory to Decision Rule: Calibrating the Noisy-Label Crossover for Vision-Language Model Weak Supervision Across Three Medical-Imaging Benchmarks

The study investigates when adding zero‑shot pseudo‑labels from vision‑language models (VLMs) to small expert‑labelled medical imaging datasets ceases to improve, and may even degrade, classifier performance. Across three benchmarks—PatchCamelyon, ISIC 2019, and NIH ChestX‑ray14—the crossover point occurs after roughly 20–50, 100, and 250–500 expert labels, respectively, with additional pseudo‑labels reducing macro‑AUC by up to 0.14 points. The analysis shows that the VLM’s labeling quality, rather than the student model size, limits performance, and a simple bias‑variance rule can predict the benefit of pseudo‑labels in most settings.

By Bruce Changlong Xu, Jose K. James, Alexander J. Ryu

llms multimodal benchmarks safety

arXiv Computation and Language

2d ago

CheXPoT: Post-Hoc Visual Verification and Correction of VLM-Generated Radiology Reports

CheXPoT is a post‑hoc visual verification framework that refines vision‑language model (VLM) generated chest radiograph reports by decomposing them into predicates, grounding each against DenseNet‑121 and Grad‑CAM evidence, and removing unsupported findings. On a held‑out set of 4,145 images, CheXPoT significantly improves precision and F1 scores across multiple CheXpert classes, boosting micro‑average precision from 19.2 % to 46.8 % and micro‑average F1 from 23.0 % to 49.4 %. The method also enhances performance for other VLM generators such as LLaVA‑Med and MAIRA‑2, demonstrating that inexpensive visual verification can correct hallucinations and improve report quality.

By Joey Guo, Rayan Ansari, Alaa Youssef

llms reinforcement-learning multimodal safety