arXiv Machine Learning
M2G-LLM: Enhancing Clinical Prediction via Multimodal Graph Reasoning and LLM Context Injection
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Oct 26
MEDMKG: Benchmarking Medical Knowledge Exploitation with Multimodal Knowledge Graph
MEDMKG is a new Medical Multimodal Knowledge Graph that combines visual data from MIMIC‑CXR with structured clinical knowledge from UMLS. The construction pipeline uses rule‑based tools and large language models for concept extraction and relationship modeling, and introduces a Neighbor‑aware Filtering algorithm to improve graph quality and compactness. Experiments on six datasets across three tasks show that MEDMKG enhances performance of twenty‑four baseline methods and four state‑of‑the‑art vision‑language backbones, providing a solid foundation for adaptive multimodal knowledge integration in medical AI.
arXiv AI
Jun 11
MLaGA: Multimodal Large Language and Graph Assistant
arXiv:2506. 02568v2 Announce Type: replace Abstract: Large Language Models (LLMs) have demonstrated substantial efficacy in advancing graph-structured data analysis.
arXiv AI
Oct 26
DIGOM: Disentangled Interaction Graphs for Outcome prediction in Multimodal healthcare
DIGOM (Disentangled Interaction Graphs for Outcome prediction in Multimodal healthcare) is a graph-based framework that jointly models intra‑patient cross‑modal, cross‑patient same‑modal, and cross‑patient cross‑modal interactions. It learns a sparse multiplex graph with patient‑adaptive connectivity through a differentiable sampling mechanism, enabling end‑to‑end learning without a fixed neighbor count. Evaluated on MIMIC‑IV and NIH‑TB datasets across five clinical prediction tasks, DIGOM achieves the highest AUROC while using fewer connections than existing multi‑interaction approaches, and shows robustness to missing modalities and task‑specific connectivity patterns.
arXiv AI
Aug 21
MKG-CARE: Case-Aware Reasoning with Multimodal Knowledge Graphs for Explainable Medical Image Diagnosis
arXiv:2605. 22547v3 Announce Type: replace-cross Abstract: Medical image diagnosis has achieved significant progress with deep learning, yet existing methods often rely on isolated visual evidence and lack the ability to effectively leverage similar cases and external knowledge.
arXiv AI
Sep 7
REFINE: LLM Refinement over Budgeted Text-Attributed Graphs for Personalized Medical Concept Representation
REFINE is a framework that refines medical concept representations by creating patient‑specific temporal graphs from a global text‑attributed knowledge graph. It uses a reinforcement learning policy to allocate a personalized graph expansion budget for each observed code, then processes the resulting graph with a heterogeneous GNN and a frozen LLM that refines representations via graph‑aware soft prompts. Experiments on MIMIC‑III and MIMIC‑IV demonstrate that REFINE consistently improves various EHR prediction backbones, surpasses strong baselines, and shows robust gains across ablation studies, KG selection, and data insufficiency scenarios.
arXiv Machine Learning
Sep 21
HERMES: Contrast-Aware Knowledge Graph Reasoning from Clinical Notes for Patient Outcome Prediction
HERMES is a graph-based framework that uses only clinical text to predict patient outcomes. It builds personalized Knowledge Graphs from clinical notes via Large‑Language‑Model‑guided extraction and Contrastive Logic Modeling, capturing temporal dynamics and treatment changes. A Graph Attention Network then synthesizes patient representations, and experiments on MIMIC‑III and MIMIC‑IV show HERMES outperforms text‑only baselines for in‑hospital mortality and 30‑day readmission prediction.