arXiv Computer Vision
Sep 14

Learning Sparse Latent Predictive Foundation Model for Multimodal Neuroimaging

The paper introduces Neuro‑JEPA, a sparse multimodal foundation model that learns unified representations of brain MRI across T1w, T2w, and FLAIR sequences using a latent predictive objective and a Mixture‑of‑Experts architecture. It was pretrained on over 1.5 million scans from 428,647 studies and systematically evaluates architectural, masking, objective, and sparsity choices for robust multimodal representation learning. Across 47 tasks from three health systems and 12 public datasets, Neuro‑JEPA consistently outperforms a simple CNN baseline, demonstrating its effectiveness for both clinical and research applications.

By Haoxu Huang, Long Chen, Jingyun Chen, Jinu Hyun, James Ryan Loftus, Kara Melmed, Daniel Orringer, Jennifer Frontera, Seena Dehkharghani, Arjun Masurkar, Narges Razavian
arXiv AI
Oct 9

SRUG: A Fusion-Driven Generator Network for Medical Image Translation

SRUG is a supervised, fusion‑driven generator network designed for paired MRI sequence synthesis. It eliminates the generator‑discriminator competition by directly learning a deterministic source‑to‑target mapping, using a residual encoder, full‑resolution stem, and a nested multi‑scale decoder to preserve structural fidelity. Experiments on BraTS 2023 and IXI demonstrate competitive reconstruction quality and cross‑dataset transferability, supporting direct supervised generation as a practical approach for medical image translation.

By Xihe Qiu, Yang Dai, Xiaoyu Tan, Sijia Li, Fenghao Sun, Lu Gan, Liang Liu