Hugging Face Trending Papers

ModaLens: Measuring Image Sensitivity in Report-Conditioned Medical VLMs

arXiv AI
Sep 15

ModaLens: Measuring Image Sensitivity in Report-Conditioned Medical VLMs

arXiv:2609.15635v1 Announce Type: cross Abstract: A radiology report can already answer a clinical question, so it is hard to tell whether a vision-language model also uses the image. ModaLens, a pai...

By Sebasti\'an Andr\'es Cajas Ord\'o\~nez, Maximin Lange, Quang Bui, Anqi Peter Li, Felipe Ocampo Osorio, Rafi Al Attrach, Kushul Reddy Palakala, Sahil Kapadia, Zakaria Laouabdia Sellami, Xinyue Zhang, Ashley Zhang, Leo Anthony Celi
arXiv Machine Learning
Oct 23

Measuring Pretraining Contamination in Public Medical Vision-Language Benchmarks

The paper investigates whether public medical vision‑language benchmarks contain data that may have leaked into the training sets of open VLMs. Using four contamination detectors, the authors find that a significant portion of images in SLAKE‑En have close neighbours in PMC‑OA‑beta, indicating source overlap rather than exact memorization. On the text side, only the exchangeability detector consistently flags potential contamination, while the other methods produce false positives and are unreliable without external baselines.

By Bruce Changlong Xu, Lan Wu, Alexander Ryu
arXiv Computation and Language
Oct 16

Spurious Attention Heads in Vision Transformers: Mechanistic Identification of Annotation-Driven Bias in Medical Chest X-Ray Analysis

The study identifies a specific attention head (L8H10) in Vision Transformers that reads annotation text on chest X‑ray images instead of lung pathology, affecting 62% of COVID‑19 images. Causal inpainting experiments confirm that the head’s behavior is driven by annotation content, and removing the head increases lung attention by 1.52×. This spurious annotation‑attending behavior is consistent across multiple ViT architectures and datasets, persists after full fine‑tuning, and is invisible to standard output‑level evaluation, highlighting the need for mechanistic auditing.

By Joshua Muhumuza, Cornelius Kalenzi