MultiSigBERT is a unified framework that performs multimodal sequential survival modeling in oncology by integrating narrative clinical reports, numerical measurements, and structured variables. The method converts free-text reports into sentence embeddings, compresses them with modality-specific PCA, and concatenates them with structured covariates to create joint temporal trajectories. These trajectories are encoded using the Signature transform from Rough Paths theory, and the resulting high-dimensional features are fed into a LASSO-regularized Cox model, achieving a concordance index of 0.743 on an independent test set of over 2,500 patients.
By Paul Minchella, St\'ephane Chr\'etien, Guillaume Metzler, Lo\"ic Verlingue, R\'emi Vaucher
arXiv:2609.14072v1 Announce Type: new
Abstract: Large Language Models (LLMs) have displayed impressive capabilities in handling tasks that require few demonstration examples, making them effective fe...
By H Mathavan, H Liu
The paper presents a newly curated, multi-center, multi-modal, and longitudinal lung cancer dataset comprising 1,365 patients with whole-slide images, CT scans, PET scans, structured clinical data, transcriptomics, and follow-up information. The dataset features substantial, non-uniform missingness across modalities, making it ideal for evaluating robust multi-modal fusion strategies. Benchmarks on 12‑month overall survival, disease‑specific survival, and longitudinal hazard prediction demonstrate that integrating complementary modalities consistently outperforms uni-modal approaches, even under severe missing data.
By Rita Cordeiro Mendes, Maria Rita Fonseca Verdelho, Carlos Santiago, Catarina Barata
A causal multi-modal AI model was developed to predict personalized chemosensitivity in breast cancer patients using routine pathology and clinical data. Trained on 9,141 patients from nine countries and validated on 1,994 patients from three countries, the model produced treatment-specific recurrence probabilities with near-perfect calibration and strong prognostic discrimination over 5- and 10-year horizons. It outperformed existing recurrence-score tests and could reduce chemotherapy prescriptions by 30% while maintaining recurrence-free rates, with predictive performance also transferring to non-breast cancers.
By Dhruva Biswas, Jeroen Berrevoets, Alec McClean, Linus Bao, Jungkyu Park, Ken G. Zeng, Joseph Cappadona, Cerise Tang, Chuwen Liu, Bartosz Machura, Yin Wu, Valerie Speirs, Hatem Soliman, Rohit Bhargava, Sheheryar Kabraji, Thaer Khoury, David Page, Brian Piening, Carlo Bifulco, Claudia Meurs, Pieter Westenend, Sylvie Chabaud, Jerome Lemonnier, Paul H. Cottu, Florence Dalenc, Fabrice Andre, Frederique Madeleine Penault-Llorca, Thomas Bachelot, Frederick Howard, Francisco J. Esteva, Kevin Kalinsky, Lajos Pusztai, Jan Witowski, Krzysztof J. Geras
arXiv:2607. 16233v1 Announce Type: cross Abstract: Integrating heterogeneous genomic and clinical modalities for joint cancer subtype classification and survival prediction remains a key challenge in precision oncology.
By Suxing Liu Byungwon Min
arXiv:2607. 15380v1 Announce Type: cross Abstract: Electronic health records combine free-text clinical narratives with structured measurements such as vital signs, laboratory values, and comorbidities.
By Ajay Madhavan Ravichandran, Bilgin Osmandoja, Klemens Budde, Klaus Netter, Tobias Strapatsas, Aljoscha Burchardt, Sebastian M\"oller, Roland Roller