arXiv Computer Vision By Yue Zhuo, Jingjia Yan, Fangyu Liu, Lingjun Liu, Zongjun Chen, Zhisheng Luo, Zhonghua Xie

GAMF-Net: Grouped Availability-Aware Multimodal Fusion for Smartphone Skin Lesion Classification

Read the original on arXiv Computer Vision →

GAMF-Net is a grouped availability‑aware multimodal fusion network designed for smartphone skin lesion classification. It divides ten clinical variables into three semantic groups—demographics, history‑related risk, and lesion presentation—and encodes each group with field values and availability masks. The image branch supplies base logits while each clinical group contributes image‑conditioned residual evidence; only available groups are fused with equal weight, and the model defaults to image‑only when no clinical data are present. Evaluated on PAD‑UFES‑20, GAMF-Net achieves an accuracy of 74.17 % and outperforms image‑only and other multimodal baselines, with ablation studies highlighting the importance of clinical information and constrained equal‑weight fusion.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv AI
Sep 11

OmniMed-FL: A Robust Multimodal Federated Learning Framework for Clinical Diagnosis

OmniMed‑FL is a multimodal federated learning framework that fuses chest radiographs and synthetic patient notes to classify five clinical conditions. The study benchmarks eight fusion strategies, three initializations, and four missing‑text imputation rules across 3–20 hospital clients under non‑IID Dirichlet partitioning, showing that federated approaches (FedAvg, FedProx, SCAFFOLD‑AdamW) outperform local‑only training. Multimodal fusion consistently improves performance, achieving macro‑F1 scores up to 0.956 on the synthetic corpus and 0.906 on the radiograph corpus.

By Ayush Debnath, Ruelia Saha, Sudip Misra
arXiv Machine Learning
Sep 4

Beyond Blur: A Semantic Tri-view Pipeline for Teledermatology Gradability via Skin Micro-relief

The paper introduces the Semantic Tri-view Pipeline, an interpretable system that automatically screens teledermatology photographs for gradability by analyzing epidermal micro-relief across up to three smartphone views. It uses a lightweight DeepLabV3+ model to segment micro-relief fidelity and aggregates the resulting spatial masks with logistic regression, leveraging viewpoint redundancy to improve robustness. Evaluated on the SCIN dataset, the approach raises the AUC from 0.81 to 0.96 on optically clear cases, offering real‑time, privacy‑by‑design feedback to filter ungradable photo sets before clinician review.

By Robert Engel
arXiv AI
Sep 3

Disease Burden over Skin Tone: Decomposing the Dermatology-AI Generalization Gap

The study examines why dermatology AI models, largely trained on light‑skinned, cancer‑focused images, perform poorly when applied to diverse patient populations. By comparing a cancer‑trained baseline, two dermatology foundation models, and a general‑purpose vision model on tone‑stratified and disease‑shifted datasets, the authors find that disease‑distribution shift, rather than skin‑tone underrepresentation, is the primary cause of generalization failure. Representation analysis shows that cancer‑specialized features lack transferable structure, while dermatology‑pretrained features maintain stronger clustering, and lightweight adaptation with about ten labeled examples per category can recover most performance.

By Nirajan Kunwor, Sanjaya Poudel, Quoc-Huy Trinh, Jahidul Arafat, Sunil Kumar Gaire