arXiv AI By Sebasti\'an Andr\'es Cajas Ord\'o\~nez, Maximin Lange, Quang Bui, Anqi Peter Li, Felipe Ocampo Osorio, Rafi Al Attrach, Kushul Reddy Palakala, Sahil Kapadia, Zakaria Laouabdia Sellami, Xinyue Zhang, Ashley Zhang, Leo Anthony Celi

ModaLens: Measuring Image Sensitivity in Report-Conditioned Medical VLMs

Read the original on arXiv AI →

The Flow has not summarised this story yet — read it at arXiv AI.

arXiv Machine Learning
Oct 23

Measuring Pretraining Contamination in Public Medical Vision-Language Benchmarks

The paper investigates whether public medical vision‑language benchmarks contain data that may have leaked into the training sets of open VLMs. Using four contamination detectors, the authors find that a significant portion of images in SLAKE‑En have close neighbours in PMC‑OA‑beta, indicating source overlap rather than exact memorization. On the text side, only the exchangeability detector consistently flags potential contamination, while the other methods produce false positives and are unreliable without external baselines.

By Bruce Changlong Xu, Lan Wu, Alexander Ryu