arXiv AI
Jun 11
MLaGA: Multimodal Large Language and Graph Assistant
arXiv:2506. 02568v2 Announce Type: replace Abstract: Large Language Models (LLMs) have demonstrated substantial efficacy in advancing graph-structured data analysis.
M2G-LLM: Enhancing Clinical Prediction via Multimodal Graph Reasoning and LLM Context Injection
Related stories
arXiv AI
Aug 21
MKG-CARE: Case-Aware Reasoning with Multimodal Knowledge Graphs for Explainable Medical Image Diagnosis
arXiv:2605. 22547v3 Announce Type: replace-cross Abstract: Medical image diagnosis has achieved significant progress with deep learning, yet existing methods often rely on isolated visual evidence and lack the ability to effectively leverage similar cases and external knowledge.
arXiv AI
Sep 7
REFINE: LLM Refinement over Budgeted Text-Attributed Graphs for Personalized Medical Concept Representation
REFINE is a framework that refines medical concept representations by creating patient‑specific temporal graphs from a global text‑attributed knowledge graph. It uses a reinforcement learning policy to allocate a personalized graph expansion budget for each observed code, then processes the resulting graph with a heterogeneous GNN and a frozen LLM that refines representations via graph‑aware soft prompts. Experiments on MIMIC‑III and MIMIC‑IV demonstrate that REFINE consistently improves various EHR prediction backbones, surpasses strong baselines, and shows robust gains across ablation studies, KG selection, and data insufficiency scenarios.
arXiv Machine Learning
2d ago
HERMES: Contrast-Aware Knowledge Graph Reasoning from Clinical Notes for Patient Outcome Prediction
HERMES is a graph-based framework that uses only clinical text to predict patient outcomes. It builds personalized Knowledge Graphs from clinical notes via Large‑Language‑Model‑guided extraction and Contrastive Logic Modeling, capturing temporal dynamics and treatment changes. A Graph Attention Network then synthesizes patient representations, and experiments on MIMIC‑III and MIMIC‑IV show HERMES outperforms text‑only baselines for in‑hospital mortality and 30‑day readmission prediction.
arXiv AI
Aug 6
Patients-like-me: A Variational LM--GNN Framework for Explainable Clinical Prediction
arXiv:2608. 04193v1 Announce Type: cross Abstract: Language models (LMs) offer strong textual representations for electronic health records (EHRs), but they encode patient sequences in isolation and provide limited explainability.
arXiv Machine Learning
Jun 5
HoT-SSM:Higher-order Temporal Knowledge Graph Reasoning with State Space Models for Health Care
arXiv:2606. 05994v1 Announce Type: new Abstract: Medical knowledge graphs (MKGs) infused with clinical knowledge have been increasingly used to model electronic health records (EHRs) to support interpretable predictions in healthcare domain.
arXiv AI
Jun 30
PromptGNN-sim: Deep Fusion and Alignment of GNN and LLMs for Text-Attributed Graph Learning
arXiv:2606. 30291v1 Announce Type: new Abstract: Text-Attributed Graphs (TAGs) combine textual semantics with graph structure and are central to many graph learning tasks.
arXiv Machine Learning
Jun 30
GLIP: Graph and LLM Joint Pretraining for Graph-Level Tasks
arXiv:2606. 29773v1 Announce Type: new Abstract: Graphs are widely used to model relational systems, with applications in domains such as social networks, finance, and biomedicine.
arXiv Machine Learning
Jul 20
Toward Federated Multimodal Graph Foundation Models: A Topology-Aware Multimodal Alignment Framework
arXiv:2607. 15687v1 Announce Type: new Abstract: Multimodal-attributed graphs (MAGs), whose nodes carry modalities such as images and text alongside topological structure, now pervade applications including social platforms, e-commerce, and biomedical networks, offering richer semantic signals than single-modality graphs.
arXiv AI
Aug 28
MedFG-VQA: Low-Frequency Memory and Graph Attention for Lightweight Medical VQA
MedFG-VQA is a lightweight medical visual question answering framework that uses a memory bank to enhance low‑frequency DCT features and graph‑enhanced cross‑attention for visual‑textual alignment. It introduces Frequency‑Memory Fusion to retrieve and fuse low‑frequency information from a learnable memory bank, and Graph‑Aware Cross‑Attention to refine cross‑modal features via graph convolution. The authors also create SynMed‑VQA, a synthetic dataset of over 2 million QA pairs across nine imaging modalities, and show that MedFG‑VQA matches or outperforms larger models on several biomedical VQA benchmarks while keeping computational costs low.
arXiv AI
Jun 3
ChatHealthAI: Aligning Electronic Health Record Representations with Large Language Models for Grounded Clinical Reasoning
arXiv:2606. 02802v1 Announce Type: new Abstract: Large language models (LLMs) exhibit strong natural-language reasoning abilities for clinical decision support, but struggle to effectively model structured longitudinal electronic health records (EHRs).
arXiv Machine Learning
Sep 15
Multi-Modal Tumor Survival Prediction via Graph-Guided Mixture of Experts
arXiv:2609.14072v1 Announce Type: new Abstract: Large Language Models (LLMs) have displayed impressive capabilities in handling tasks that require few demonstration examples, making them effective fe...