arXiv AI
Sep 21

MIST: Multimodal Survival Prediction with Genomic-Guided Histology Attention

MIST is a multimodal survival prediction framework that fuses whole-slide images and genomic profiles by representing genomic features as tokens that query histology context tokens derived from a foundation model. The architecture enriches molecular information with histology context before survival prediction, avoiding late-stage merging of separately encoded modalities. Training incorporates discrete-time survival prediction, genomic feature masking, WSI dropout, and contrastive alignment, and demonstrates improved external C-index across colon, renal, lung, and glioblastoma cohorts compared to standard fusion baselines.

By Muhammet Sami Yavuz, Sabri Mustafa Kahya, Richard R. Chen, Jana Lipkova, Benedikt Wiestler
arXiv Machine Learning
Sep 3

Morphology signal in whole slide image foundation models can automatically triage slides

The paper introduces a pipeline that uses publicly available whole slide image foundation models (FMs) to automatically triage slides by ranking them based on zero‑shot classification predictions. This approach accurately identifies slides containing the most tumor, achieving top‑2 ranking for patients with up to 43 slides across multiple datasets. The study also proposes a ranked evaluation framework to benchmark FM performance in slide triage.

By Ayushi Sinha, Shashank Yadav, Benjamin Holmes, Pravat Das, Aaron W. Bogan, James S. Lewis Jr., Santiago Romero-Brufau, Andrew Y. K. Foong, Scott H. Kaufmann, Kathryn M. Van Abel, David M. Routman, Michael R. Lucas
arXiv Computer Vision
Sep 7

Real-World Multi-Modal and Longitudinal Lung Cancer Dataset

The paper presents a newly curated, multi-center, multi-modal, and longitudinal lung cancer dataset comprising 1,365 patients with whole-slide images, CT scans, PET scans, structured clinical data, transcriptomics, and follow-up information. The dataset features substantial, non-uniform missingness across modalities, making it ideal for evaluating robust multi-modal fusion strategies. Benchmarks on 12‑month overall survival, disease‑specific survival, and longitudinal hazard prediction demonstrate that integrating complementary modalities consistently outperforms uni-modal approaches, even under severe missing data.

By Rita Cordeiro Mendes, Maria Rita Fonseca Verdelho, Carlos Santiago, Catarina Barata