arXiv Machine Learning

Towards Unified Multimodal Graph Foundation Model: A Bridge-Router-Adapter Based Approach

arXiv Machine Learning
Jul 20

Toward Federated Multimodal Graph Foundation Models: A Topology-Aware Multimodal Alignment Framework

arXiv:2607. 15687v1 Announce Type: new Abstract: Multimodal-attributed graphs (MAGs), whose nodes carry modalities such as images and text alongside topological structure, now pervade applications including social platforms, e-commerce, and biomedical networks, offering richer semantic signals than single-modality graphs.

By Xunkai Li, Guohao Fu, Yuming Ai, Zhengyu Wu, Hongchao Qin, Rong-Hua Li, Guoren Wang
arXiv Machine Learning
Aug 4

Towards Effective Federated Multimodal Graph Learning via Navigating Multifaceted Heterogeneity

arXiv:2608. 00623v1 Announce Type: new Abstract: Multimodal-attributed graphs (MAGs), where nodes carry heterogeneous semantic content across multiple modalities while edges encode relational dependencies, have been widely adopted across diverse domains.

By Yinlin Zhu, Di Wu, Yi Zhang, Xunkai Li, Wang Luo, Wei-Jin Huang, Miao Hu, Guocong Quan
arXiv Machine Learning
Oct 5

Toward Omni Multimodal Graph Foundation Model: A Topology-Driven Binding Approach

The paper introduces GraphBind, a topology-driven method for multimodal graph foundation models that binds heterogeneous node modalities into a unified shared space using graph topology. By leveraging stable graph structure to organize self and neighborhood semantics, GraphBind adapts this integrated space for both discriminative and generative tasks. Experiments against 11 baselines show that GraphBind outperforms them, achieving up to 28.1% relative improvement on key tasks.

By Xunkai Li, Chenxi Wan, Yinlin Zhu, Wang Luo, Hongchao Qin, Rong-Hua Li, Guoren Wang
arXiv Machine Learning
Oct 3

Coupling Perception and Reasoning in Federated Multimodal Graph Foundation Models

The paper introduces FedCORE, a federated adaptation framework for multimodal graph foundation models that jointly optimizes perception (Encoder) and reasoning (GNN) modules via a shared low‑dimensional latent state. Unlike prior methods that freeze the Encoder, FedCORE allows both components to adapt together, addressing the dependency between multimodal evidence extraction and graph‑based relational reasoning. Experiments show that FedCORE significantly narrows the Encoder–GNN pairing gap, achieving an 80.7% reduction compared to independent joint adaptation.

By Zekai Chen, Xun Wu, Hailin Zhang, Xunkai Li, Yu Liu, Kairui Yang, Muyan Huang, Xuaner Chen, Rong-Hua Li, Guoren Wang
arXiv Machine Learning
Sep 29

Supervised Deep Multimodal Matrix Factorization for Interpretable Brain Network Analysis

The paper introduces Supervised Deep Multimodal Matrix Factorization (SD3MF), an interpretable framework that extends Symmetric Nonnegative Matrix Tri-Factorization to supervised prediction across populations of multimodal brain graphs. SD3MF learns deep hierarchical factorizations for each modality and a shared latent representation, jointly optimizing graph reconstruction and prediction while enabling data-driven multimodal fusion. Experiments on multimodal connectome datasets demonstrate that SD3MF outperforms strong deep learning baselines such as CNNs and GNNs, providing biologically interpretable insights through community-level interaction matrices.

By Amjad Seyedi, Lifang He, Songlin Zhao, Akwum Onwunta, Nicolas Gillis
arXiv Machine Learning
Jul 22

One Model, Many Graphs: Learning over Attributed Graphs across Heterogeneous Modalities with Vision-Language Models

arXiv:2607. 19128v1 Announce Type: new Abstract: Vision-language models (VLMs) provide a unified representation space for textual and visual information, yet their potential as general-purpose backbones for graph-structured data remains largely unexplored.

By Jiayi Yang, Yifang Chen, Yuanfu Sun, Jiajin Liu, Qiaoyu Tan
arXiv AI
Oct 26

DIGOM: Disentangled Interaction Graphs for Outcome prediction in Multimodal healthcare

DIGOM (Disentangled Interaction Graphs for Outcome prediction in Multimodal healthcare) is a graph-based framework that jointly models intra‑patient cross‑modal, cross‑patient same‑modal, and cross‑patient cross‑modal interactions. It learns a sparse multiplex graph with patient‑adaptive connectivity through a differentiable sampling mechanism, enabling end‑to‑end learning without a fixed neighbor count. Evaluated on MIMIC‑IV and NIH‑TB datasets across five clinical prediction tasks, DIGOM achieves the highest AUROC while using fewer connections than existing multi‑interaction approaches, and shows robustness to missing modalities and task‑specific connectivity patterns.

By Hakima Laribi, Olivier Lefebvre, Dan Poenaru, Martin Valli\`eres