arXiv Computer Vision

HGASurv: Hierarchical Granularity-Aware Multimodal Integration for Survival Prediction

arXiv AI
Sep 21

MIST: Multimodal Survival Prediction with Genomic-Guided Histology Attention

MIST is a multimodal survival prediction framework that fuses whole-slide images and genomic profiles by representing genomic features as tokens that query histology context tokens derived from a foundation model. The architecture enriches molecular information with histology context before survival prediction, avoiding late-stage merging of separately encoded modalities. Training incorporates discrete-time survival prediction, genomic feature masking, WSI dropout, and contrastive alignment, and demonstrates improved external C-index across colon, renal, lung, and glioblastoma cohorts compared to standard fusion baselines.

By Muhammet Sami Yavuz, Sabri Mustafa Kahya, Richard R. Chen, Jana Lipkova, Benedikt Wiestler
arXiv AI
Sep 10

Semantic Context-aware mOdality fUsion Transformer (SCOUT): A Context-Aware Multimodal Transformer for Concept-Grounded Pathology Report Generation

SCOUT is a concept‑grounded multimodal transformer that generates whole‑slide pathology reports by integrating local histological patterns, whole‑slide context, and expert‑curated diagnostic concepts. It uses evolving visual representations and recursively updated slide‑ and concept‑conditioned representations, with separate attention pathways during decoding that are fused adaptively for each token. Evaluated on TCGA‑BRCA, HistAI, and REG‑2025, SCOUT outperformed existing methods, improving BLEU, METEOR, and ROUGE‑L scores and raising the Clinical Report Quality Score on REG‑2025.

By Suryakant Singh, Saarthak Kapse, Joel Saltz, Prateek Prasanna
arXiv AI
Sep 1

Towards Accurate and Lightweight Peripheral Neuroblastic Tumor Diagnosis via Contrastive Multi-scale Pathological Image Analysis

The paper introduces CoPath, a lightweight framework for diagnosing peripheral neuroblastic tumors (pNTs) from whole-slide images. CoPath combines CoHisNet, a multi‑scale feature‑fusion network that replaces traditional MLPs with Kolmogorov‑Arnold Network layers for efficient nonlinear modeling, and PathVote, which aggregates patch‑level predictions using pathology‑informed priors. Experiments on a private pNT cohort and the public BreakHis dataset show that CoPath matches or surpasses existing classifiers while reducing computational complexity.

By Zhu Zhu, Shuo Jiang, Jingyuan Zheng, Yawen Li, Yifei Chen, Manli Zhao, Weizhong Gu, Feiwei Qin, Jinhu Wang, Gang Yu
arXiv Machine Learning
Jul 24

HierarchicalDAEW: Domain-Aware Edge-Weighted Graph Convolution with Evidential Uncertainty for Multi-Section Spatial Gene Expression Prediction from H&E Histology

arXiv:2607. 20896v1 Announce Type: new Abstract: Spatial transcriptomics assays remain costly and technically demanding, restricting transcriptome-wide profiling to specialist settings and preventing routine clinical deployment.

By Kritanu Chattopadhyay, Soumya Chatterjee, Ondrej Krejcar, Debotosh Bhattacharjee
arXiv Machine Learning
Jun 5

HEIST: A Graph Foundation Model for Spatial Transcriptomics and Proteomics Data

arXiv:2506. 11152v4 Announce Type: replace-cross Abstract: Single-cell transcriptomics and proteomics have become a great source for data-driven insights into biology, enabling the use of advanced deep learning methods to understand cellular heterogeneity and gene expression at the single-cell level.

By Hiren Madhu, Jo\~ao Felipe Rocha, Tinglin Huang, Siddharth Viswanath, Smita Krishnaswamy, Rex Ying