arXiv:2609.15635v1 Announce Type: cross
Abstract: A radiology report can already answer a clinical question, so it is hard to tell whether a vision-language model also uses the image. ModaLens, a pai...
By Sebasti\'an Andr\'es Cajas Ord\'o\~nez, Maximin Lange, Quang Bui, Anqi Peter Li, Felipe Ocampo Osorio, Rafi Al Attrach, Kushul Reddy Palakala, Sahil Kapadia, Zakaria Laouabdia Sellami, Xinyue Zhang, Ashley Zhang, Leo Anthony Celi
The paper investigates whether public medical vision‑language benchmarks contain data that may have leaked into the training sets of open VLMs. Using four contamination detectors, the authors find that a significant portion of images in SLAKE‑En have close neighbours in PMC‑OA‑beta, indicating source overlap rather than exact memorization. On the text side, only the exchangeability detector consistently flags potential contamination, while the other methods produce false positives and are unreliable without external baselines.
By Bruce Changlong Xu, Lan Wu, Alexander Ryu
arXiv:2606. 02276v1 Announce Type: cross Abstract: Vision-language models (VLMs) trained on paired chest radiographs and radiology reports learn a shared embedding space that can preserve instance-level image-report correspondence.
By Soroosh Tayebi Arasteh, Mahshad Lotfinia, Sven Nebelung, Daniel Truhn
arXiv:2606. 17710v1 Announce Type: cross Abstract: Medical vision-language models report strong chest radiograph accuracy, and this is increasingly read as evidence that they use the image.
By Mahshad Lotfinia, Sebastian Ziegelmayer, Lisa Adams, Daniel Truhn, Andreas Maier, Soroosh Tayebi Arasteh
arXiv:2604. 08941v2 Announce Type: replace Abstract: Medical Vision-Language Models (VLMs) answering binary presence questions on chest radiographs can fail in two linked ways: they are confidently wrong, and they change answers when a clinically equivalent question is rephrased.
By Binesh Sadanandan, Vahid Behzadan
arXiv:2606. 10066v1 Announce Type: cross Abstract: Medical vision-language models (VLMs) are evaluated on public benchmarks whose images and question-answer pairs have been freely downloadable for years, yet reported accuracy assumes these examples were absent from pretraining.
By Bruce Changlong Xu, Lan Wu, Alexander Ryu
arXiv:2607. 25589v1 Announce Type: cross Abstract: Medical-imaging AI benchmarks combine datasets, DICOM rendering, prompts, provider APIs, automated labels, statistical code, manuscripts, and repository releases.
By Mateusz Koz{\l}owski
The study identifies a specific attention head (L8H10) in Vision Transformers that reads annotation text on chest X‑ray images instead of lung pathology, affecting 62% of COVID‑19 images. Causal inpainting experiments confirm that the head’s behavior is driven by annotation content, and removing the head increases lung attention by 1.52×. This spurious annotation‑attending behavior is consistent across multiple ViT architectures and datasets, persists after full fine‑tuning, and is invisible to standard output‑level evaluation, highlighting the need for mechanistic auditing.
By Joshua Muhumuza, Cornelius Kalenzi
arXiv:2606.15910v3 Announce Type: replace
Abstract: An answer from a vision-language model (VLM) to a medical image question can be accepted without review only under a threshold bounding the error o...
By Reza Khanmohammadi, Kundan Thind, Mohammad M. Ghassemi
arXiv:2609.00868v1 Announce Type: cross
Abstract: Vision-language models are evaluated by aggregate accuracy on multimodal benchmarks, a practice that implicitly assumes the model uses its visual inp...
By Genpei Zhang
arXiv:2609. 31450v1 Announce Type: cross Abstract: Medical vision-language model (VLM) post-training is commonly evaluated through answer accuracy.
By Wang Jingxin
arXiv:2607. 01973v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) are increasingly applied in medical tasks such as pathology description, report generation, and visual question answering.
By Sofiane Ouaari, Kevin Vorwalder, Nico Pfeifer