The paper evaluates how well generalist and dermatology-specific machine learning models perform on diverse skin lesion datasets, including dermoscopic images and smartphone photographs. It benchmarks a range of architectures—general-purpose vision-language models, foundation models, and task-specific dermatology classifiers—under conditions of distribution shift, modality change, and demographic variability. The study quantifies the performance gap between current state‑of‑the‑art models and the robustness needed for safe, equitable clinical deployment.
By Emanoel dos Santos, Kelvin Cunha, Rodrigo Mota, Fabio Papais, Thales Bezerra, Natalia Lopes, Erico Medeiros, Shirley Cruz, Jessica Araujo, Paulo Borba, Tsang Ing Ren
arXiv:2606. 06224v1 Announce Type: cross Abstract: Explanations of multiple instance learning (MIL) models are widely used for validation and discovery in digital histopathology.
By Yanqing Luo (Berlin Institute for the Foundations of Learning and Data, Berlin, Germany, Machine Learning Group, Technische Universit\"at Berlin, Berlin, Germany), Julius Hense (Berlin Institute for the Foundations of Learning and Data, Berlin, Germany, Machine Learning Group, Technische Universit\"at Berlin, Berlin, Germany), Niklas Preni{\ss}l (Institute of Pathology, Charit\'e Universit\"atsmedizin, Berlin, Germany, Berlin Institute of Health at Charit\'e -- Universit\"atsmedizin Berlin, BIH Biomedical Innovation Academy, BIH Charit\'e Digital Clinician Scientist Program, Berlin, Germany), Andreas Mock (Institute of Pathology, Ludwig Maximilian University of Munich, Munich, Germany, Division of Translational Medical Oncology, DKFZ, Heidelberg, Germany, NCT Heidelberg, Heidelberg, Germany, German Cancer Consortium), Klaus-Robert M\"uller (Berlin Institute for the Foundations of Learning and Data, Berlin, Germany, Machine Learning Group, Technische Universit\"at Berlin, Berlin, Germany, Department of Artificial Intelligence, Korea University, Seoul, Korea, Max-Planck Institute for Informatics, Saarbr\"ucken, Germany), Thomas Schnake (Department of Chemistry, Chemical Physics Theory Group, University of Toronto, Canada, Vector Institute for Artificial Intelligence, Toronto, Canada, Acceleration Consortium, University of Toronto, Canada), Mina Jamshidi Idaji (Berlin Institute for the Foundations of Learning and Data, Berlin, Germany, Machine Learning Group, Technische Universit\"at Berlin, Berlin, Germany)
The study examines why dermatology AI models, largely trained on light‑skinned, cancer‑focused images, perform poorly when applied to diverse patient populations. By comparing a cancer‑trained baseline, two dermatology foundation models, and a general‑purpose vision model on tone‑stratified and disease‑shifted datasets, the authors find that disease‑distribution shift, rather than skin‑tone underrepresentation, is the primary cause of generalization failure. Representation analysis shows that cancer‑specialized features lack transferable structure, while dermatology‑pretrained features maintain stronger clustering, and lightweight adaptation with about ten labeled examples per category can recover most performance.
By Nirajan Kunwor, Sanjaya Poudel, Quoc-Huy Trinh, Jahidul Arafat, Sunil Kumar Gaire