ModaLens: Measuring Image Sensitivity in Report-Conditioned Medical VLMs
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
arXiv:2609.15635v1 Announce Type: cross Abstract: A radiology report can already answer a clinical question, so it is hard to tell whether a vision-language model also uses the image. ModaLens, a pai...
The paper investigates whether public medical vision‑language benchmarks contain data that may have leaked into the training sets of open VLMs. Using four contamination detectors, the authors find that a significant portion of images in SLAKE‑En have close neighbours in PMC‑OA‑beta, indicating source overlap rather than exact memorization. On the text side, only the exchangeability detector consistently flags potential contamination, while the other methods produce false positives and are unreliable without external baselines.
arXiv:2606. 02276v1 Announce Type: cross Abstract: Vision-language models (VLMs) trained on paired chest radiographs and radiology reports learn a shared embedding space that can preserve instance-level image-report correspondence.
arXiv:2606. 17710v1 Announce Type: cross Abstract: Medical vision-language models report strong chest radiograph accuracy, and this is increasingly read as evidence that they use the image.
arXiv:2604. 08941v2 Announce Type: replace Abstract: Medical Vision-Language Models (VLMs) answering binary presence questions on chest radiographs can fail in two linked ways: they are confidently wrong, and they change answers when a clinically equivalent question is rephrased.
arXiv:2606. 10066v1 Announce Type: cross Abstract: Medical vision-language models (VLMs) are evaluated on public benchmarks whose images and question-answer pairs have been freely downloadable for years, yet reported accuracy assumes these examples were absent from pretraining.