arXiv AI

ModaLens: Measuring Image Sensitivity in Report-Conditioned Medical VLMs

Hugging Face Trending Papers

Sep 14

ModaLens: Measuring Image Sensitivity in Report-Conditioned Medical VLMs

A radiology report can already answer a clinical question, so it is hard to tell whether a vision-language model also uses the image. ModaLens, a paired image-swap audit, measures how report availabil...

llms multimodal

arXiv AI

Jun 2

Cross-modal linkage risk in clinical vision-language models

arXiv:2606. 02276v1 Announce Type: cross Abstract: Vision-language models (VLMs) trained on paired chest radiographs and radiology reports learn a shared embedding space that can preserve instance-level image-report correspondence.

By Soroosh Tayebi Arasteh, Mahshad Lotfinia, Sven Nebelung, Daniel Truhn

llms rag fine-tuning multimodal benchmarks safety

arXiv AI

Jun 17

Vision-language models for chest radiography do not always need the image

arXiv:2606. 17710v1 Announce Type: cross Abstract: Medical vision-language models report strong chest radiograph accuracy, and this is increasingly read as evidence that they use the image.

By Mahshad Lotfinia, Sebastian Ziegelmayer, Lisa Adams, Daniel Truhn, Andreas Maier, Soroosh Tayebi Arasteh

llms multimodal benchmarks

arXiv Machine Learning

Aug 12

Predictive Entropy as a Joint Screen for Error and Paraphrase Instability in Medical Vision-Language Models

arXiv:2604. 08941v2 Announce Type: replace Abstract: Medical Vision-Language Models (VLMs) answering binary presence questions on chest radiographs can fail in two linked ways: they are confidently wrong, and they change answers when a clinically equivalent question is rephrased.

By Binesh Sadanandan, Vahid Behzadan

llms multimodal

arXiv AI

Jun 10

A Controlled Audit of Pretraining Contamination in Public Medical Vision-Language Benchmarks

arXiv:2606. 10066v1 Announce Type: cross Abstract: Medical vision-language models (VLMs) are evaluated on public benchmarks whose images and question-answer pairs have been freely downloadable for years, yet reported accuracy assumes these examples were absent from pretraining.

By Bruce Changlong Xu, Lan Wu, Alexander Ryu

llms multimodal benchmarks

arXiv AI

Jul 31

Forensic Reproducibility Audit of a Radiology Vision-Language Model Benchmark: From Intended Protocol to Released Artifact

arXiv:2607. 25589v1 Announce Type: cross Abstract: Medical-imaging AI benchmarks combine datasets, DICOM rendering, prompts, provider APIs, automated labels, statistical code, manuscripts, and repository releases.

By Mateusz Koz{\l}owski

llms multimodal benchmarks

arXiv Machine Learning

Sep 2

The Visual Insensitivity Gap: Diagnosing When Vision-Language Models Fail to Use Visual Evidence

arXiv:2609.00868v1 Announce Type: cross Abstract: Vision-language models are evaluated by aggregate accuracy on multimodal benchmarks, a practice that implicitly assumes the model uses its visual inp...

By Genpei Zhang

llms multimodal benchmarks

arXiv AI

Jul 3

Assessing VLM Reliability for Medical Image Quality Evaluation Under Corruption and Bias

arXiv:2607. 01973v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) are increasingly applied in medical tasks such as pathology description, report generation, and visual question answering.

By Sofiane Ouaari, Kevin Vorwalder, Nico Pfeifer

llms rag nlp multimodal benchmarks safety

arXiv AI

Aug 11

JustLLMGRPO: Radiographic Control for Chest X-Ray Generation

arXiv:2608. 08046v1 Announce Type: new Abstract: Text-conditioned chest X-ray generation aims to synthesize realistic radiographs that faithfully depict specified findings.

By Pengxiang Cai, Xiaohan Li, Anglin Liu, Qingyuan Zeng, Zexun Li, Jintai Chen

llms fine-tuning benchmarks safety

arXiv AI

5d ago

From Reward Signal to Visual Utility: A Controlled Audit of Medical VLM Post-Training

arXiv:2609. 31450v1 Announce Type: cross Abstract: Medical vision-language model (VLM) post-training is commonly evaluated through answer accuracy.

By Wang Jingxin

llms fine-tuning multimodal

arXiv AI

3d ago

It's Not What the Image Shows: Irrelevant Context Destabilises VLM Judges Without Informing Them

arXiv:2609.37863v1 Announce Type: cross Abstract: Vision-language models (VLMs) are increasingly used in place of human annotators, making it important that substitutability tests reflect the model r...

By Nagham Omar, Mahmoud Jabarin, Kinan Ibraheem, Lotem Peled-Cohen

llms multimodal

arXiv Machine Learning

Aug 31

Prompts Without Evidence: How Neuroimaging Mentions Shift Clinical Vision-Language Model Predictions

arXiv:2603.28387v3 Announce Type: replace-cross Abstract: Trustworthy clinical AI must use real evidence and avoid relying on surface-level artifacts. We evaluate 12 open-weight vision-language model...

By Doan Nam Long Vu, Simone Balloccu

llms multimodal safety

Related stories