A radiology report can already answer a clinical question, so it is hard to tell whether a vision-language model also uses the image. ModaLens, a paired image-swap audit, measures how report availabil...
The paper investigates whether public medical vision‑language benchmarks contain data that may have leaked into the training sets of open VLMs. Using four contamination detectors, the authors find that a significant portion of images in SLAKE‑En have close neighbours in PMC‑OA‑beta, indicating source overlap rather than exact memorization. On the text side, only the exchangeability detector consistently flags potential contamination, while the other methods produce false positives and are unreliable without external baselines.
By Bruce Changlong Xu, Lan Wu, Alexander Ryu
arXiv:2606. 02276v1 Announce Type: cross Abstract: Vision-language models (VLMs) trained on paired chest radiographs and radiology reports learn a shared embedding space that can preserve instance-level image-report correspondence.
By Soroosh Tayebi Arasteh, Mahshad Lotfinia, Sven Nebelung, Daniel Truhn
arXiv:2606. 17710v1 Announce Type: cross Abstract: Medical vision-language models report strong chest radiograph accuracy, and this is increasingly read as evidence that they use the image.
By Mahshad Lotfinia, Sebastian Ziegelmayer, Lisa Adams, Daniel Truhn, Andreas Maier, Soroosh Tayebi Arasteh
arXiv:2604. 08941v2 Announce Type: replace Abstract: Medical Vision-Language Models (VLMs) answering binary presence questions on chest radiographs can fail in two linked ways: they are confidently wrong, and they change answers when a clinically equivalent question is rephrased.
By Binesh Sadanandan, Vahid Behzadan
arXiv:2606. 10066v1 Announce Type: cross Abstract: Medical vision-language models (VLMs) are evaluated on public benchmarks whose images and question-answer pairs have been freely downloadable for years, yet reported accuracy assumes these examples were absent from pretraining.
By Bruce Changlong Xu, Lan Wu, Alexander Ryu
The study identifies a specific attention head (L8H10) in Vision Transformers that reads annotation text on chest X‑ray images instead of lung pathology, affecting 62% of COVID‑19 images. Causal inpainting experiments confirm that the head’s behavior is driven by annotation content, and removing the head increases lung attention by 1.52×. This spurious annotation‑attending behavior is consistent across multiple ViT architectures and datasets, persists after full fine‑tuning, and is invisible to standard output‑level evaluation, highlighting the need for mechanistic auditing.
By Joshua Muhumuza, Cornelius Kalenzi
arXiv:2607. 25589v1 Announce Type: cross Abstract: Medical-imaging AI benchmarks combine datasets, DICOM rendering, prompts, provider APIs, automated labels, statistical code, manuscripts, and repository releases.
By Mateusz Koz{\l}owski
arXiv:2609.00868v1 Announce Type: cross
Abstract: Vision-language models are evaluated by aggregate accuracy on multimodal benchmarks, a practice that implicitly assumes the model uses its visual inp...
By Genpei Zhang
arXiv:2607. 01973v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) are increasingly applied in medical tasks such as pathology description, report generation, and visual question answering.
By Sofiane Ouaari, Kevin Vorwalder, Nico Pfeifer
arXiv:2610.17989v1 Announce Type: cross
Abstract: A medical vision-language model may analyze a given CT scan and correctly detect both kidneys in view. Continuing this conversation, if a user provid...
By Sadif Ahmed, Soumya Swagata Biswas, Rifat Shahriyar, Chowdhury Mohammad Rakin Haider
arXiv:2608. 08046v1 Announce Type: new Abstract: Text-conditioned chest X-ray generation aims to synthesize realistic radiographs that faithfully depict specified findings.
By Pengxiang Cai, Xiaohan Li, Anglin Liu, Qingyuan Zeng, Zexun Li, Jintai Chen