arXiv Machine Learning

Evaluation Choices Shape Biomedical ML Claims: A Pediatric Pneumonia Benchmark Case Study

arXiv Machine Learning
Jul 30

Rethinking Clinical Relevance in Chest X-ray Machine Learning: How Evaluation References Define Performance

arXiv:2607. 26333v1 Announce Type: cross Abstract: Chest X-ray (CXR) machine learning relies heavily on automated evaluation using reference standards that aim to approximate clinical judgment.

By Panagiotis Fytas, Ian Selby, Clemens Karner, Judith Babar, Simon Baker, Jake Beckford, Timothy J. Sadler, Shahab Shahipasand, Arthikkaa Thavakumar, John Li Chen, Alex Sawer, Michael Roberts, Jonathan Weir-McCall, J. H. F. Rudd, Carola-Bibiane Sch\"onlieb, Anna Korhonen, Anna Breger
arXiv Machine Learning
Aug 18

On Cross-Validation for Hyperparameter Optimization of Deep Learning Image Classifiers

arXiv:2608. 14705v1 Announce Type: cross Abstract: Hyperparameter optimization (HPO) can materially affect the performance of deep learning (DL) image classifiers, but there is little empirical guidance on how to derive the validation signal that drives it, especially for the small sample sizes common in fields such as medical imaging.

By Ljubomir Buturovic (East Palo Alto, United States)
arXiv Computer Vision
Aug 27

Label-Free Foundational Model Selection for Medical Image Classification under Distribution Shift via Pseudo Label Discrepancy

The paper introduces a label‑free method called AURCC for selecting the best foundational model for medical image classification when the target domain lacks labels. AURCC uses a pseudo‑label discrepancy computed by the SUDO framework to score models without fine‑tuning. Experiments on chest X‑ray data across three inter‑hospital shifts show that AURCC closely matches the true model ranking, outperforming simple source‑accuracy baselines especially when source data are limited.

By Juan I\~naki Larrea, Lucas Mansilla, Enzo Ferrante
arXiv AI
Aug 18

Picking the Right Image to Classify: Reliable-Input Selection in Teledermatology

arXiv:2608. 16198v1 Announce Type: cross Abstract: Dermatology models face distribution shifts in teledermatology settings, where submitted images differ from the training data in lighting, angle, distance, focus, and framing.

By Fabian Gr\"oger, Marco Weishaupt, Philippe Gottfrois, Simone Lionetti, Linda Wermelinger, Nipun Ranasekara, Ludovic Amruthalingam, Alexander A. Navarini, Marc Pouly
Hugging Face Trending Papers
Sep 4

Cross-dataset transportability of pediatric chest X-ray deep learning across three countries: discrimination, calibration, operating-point failure, and limited-label recovery

The study evaluates a deep‑learning model for pediatric pneumonia detection across three countries, testing not only discrimination but also probability calibration, fixed operating‑point transport, shortcut signals, and limited‑label recoverability. Using a DenseNet121 ensemble trained on Guangzhou data, the model achieved high internal AUROC (0.976) but performance dropped to 0.798 and 0.742 on Bangladeshi and Vietnamese datasets, respectively. Limited‑label adaptation with Platt recalibration restored sensitivity but introduced significant specificity variability, highlighting the need for comprehensive cross‑dataset evaluation.

arXiv AI
Sep 25

Med-AR: Autoregressive Vision-Language Pretraining for Long-Tailed Chest X-Ray Classification and Uncertainty-Aware Evaluation

Med-AR introduces two autoregressive vision‑language models, Med‑AR‑8B and Med‑AR‑2B, pretrained on structured radiology reports, abnormality‑focused text, and region annotations to address long‑tailed chest X‑ray classification. The models outperform existing contrastive, self‑supervised, and supervised encoders—including Med‑CLIP, CheXFound, EVA‑Base, ARK, and BioViL‑T—across PadChest, MIMIC‑CXR, and CheXpert, achieving higher mean AUROC and AUPRC for head, medium, and tail findings and lower excess area under the risk‑coverage curve. Med‑AR also demonstrates improved selective‑prediction performance, with Med‑AR‑8B raising tail‑label mean AUPRC on MIMIC‑CXR from 0.1033 to 0.1441 and Med‑AR‑2B delivering the strongest discrimination on PadChest.

By Janhavi Prabhu, Sahil, Akshay V, Shivam Shukla, Manoj Tadepalli, Preetham Putha