arXiv AI

MGDT: MLLM-Guided Diffusion Transformer with Relation-Adaptive Mixture-of-Experts for Multimodal Knowledge Graph Completion

arXiv:2607. 15592v1 Announce Type: new Abstract: Multimodal Knowledge Graph Completion (MKGC) requires inferring missing entities from structural, textual, and visual cues.

arXiv Machine Learning

Jul 14

Hyper-modal Imputation Diffusion Embedding with Dual-Distillation for Federated Multimodal Knowledge Graph Completion

arXiv:2506. 22036v2 Announce Type: replace Abstract: With the increasing multimodal knowledge privatization requirements, multimodal knowledge graphs in different institutes are usually decentralized, lacking of effective collaboration system with both stronger reasoning ability and transmission safety guarantees.

By Ying Zhang, Yu Zhao, Xuhui Sui, Baohang Zhou, Xiangrui Cai, Li Shen, Xiaojie Yuan, Dacheng Tao

rag diffusion efficiency multimodal benchmarks safety

arXiv AI

Aug 7

ViSR-KGC: Visual Subgraph Reasoning with Vision-Language Models for Multimodal Knowledge Graph Completion

arXiv:2608. 05833v1 Announce Type: new Abstract: Knowledge graph completion (KGC) aims to infer missing entities or relations from incomplete graph structures, and has evolved into multimodal knowledge graph completion (MMKGC), where entities are associated with multiple modalities such as text and images.

By Jiafan Li, Mengxue Yang, Jiaqi Zhu, Liang Chang, Ying Li, Hongan Wang

llms rag multimodal

arXiv AI

Jul 21

MMGraphRAG: Bridging Vision and Language with Interpretable Multimodal Knowledge Graphs

arXiv:2507. 20804v3 Announce Type: replace Abstract: Large Language Models (LLMs) suffer from hallucinations due to their static parametric knowledge.

By Xueyao Wan, Hang Yu

llms rag multimodal benchmarks safety

arXiv AI

Jul 21

RADD: Retrieval-Augmented Discrete Diffusion for Multi-Modal Knowledge Graph Completion

arXiv:2604. 25693v2 Announce Type: replace Abstract: Most multi-modal knowledge graph completion (MMKGC) models use one embedding scorer to conduct both retrieval over the full entity set and final link prediction.

By Guanglin Niu, Bo Li

llms rag diffusion efficiency multimodal benchmarks

arXiv Machine Learning

Jun 30

Symbiotic-MoE: Unlocking the Synergy between Generation and Understanding

arXiv:2604. 07753v2 Announce Type: replace-cross Abstract: Empowering Large Multimodal Models (LMMs) with image generation often leads to catastrophic forgetting in understanding tasks due to severe gradient conflicts.

By Xiangyue Liu, Zijian Zhang, Miles Yang, Zhao Zhong, Liefeng Bo, Ping Tan

llms diffusion multimodal benchmarks

arXiv Machine Learning

Jun 17

SMGFM: Spectral Multimodal Graph Pretraining for Multimodal-Attributed Graphs

arXiv:2606. 12867v2 Announce Type: replace Abstract: Multimodal-attributed graphs (MAGs) couple graph topology with node semantics from text, images, and other modalities.

By Zhengyu Wu, Xu Wang, Hongchao Qin, Xunkai Li, Guang Zeng, Rong-Hua Li, Guoren Wang

multimodal benchmarks safety

arXiv Machine Learning

Jun 15

Context-aware Modality-Topology Co-Alignment for Multimodal Attributed Graphs

arXiv:2606. 14172v1 Announce Type: new Abstract: Multimodal Attributed Graphs (MAGs) model real-world entities by coupling graph topology with heterogeneous attributes such as text and images.

By Sirui Zhang, Xu Wang, Zhengyu Wu, Xunkai Li, Hongchao Qin

multimodal safety

arXiv AI

Jul 14

MG$^2$-RAG: Multi-Granularity Graph for Multimodal Retrieval-Augmented Generation

arXiv:2604. 04969v2 Announce Type: replace-cross Abstract: Retrieval-Augmented Generation (RAG) mitigates hallucinations in Multimodal Large Language Models (MLLMs), yet existing systems struggle with complex cross-modal reasoning.

By Sijun Dai, Qiang Huang, Xiaoxing You, Jun Yu

llms rag multimodal benchmarks safety

arXiv AI

Jul 14

PivotMerge: Bridging Heterogeneous Multimodal Pre-training via Post-Alignment Model Merging

arXiv:2604. 22823v2 Announce Type: replace-cross Abstract: Multimodal Large Language Models (MLLMs) rely on multimodal pre-training over diverse data sources, where different datasets often induce complementary cross-modal alignment capabilities.

By Zibo Shao, Baochen Xiong, Xiaoshan Yang, Yaguang Song, Qimeng Zhang, Haifeng Chen, Changsheng Xu

llms fine-tuning multimodal benchmarks safety

arXiv AI

Jun 17

Correct When Paired, Wrong When Split: Decoupling and Editing Modality-Specific Neurons in MLLMs

arXiv:2606. 17057v1 Announce Type: cross Abstract: Although Knowledge Editing provides an efficient mechanism for updating the knowledge of Multimodal Large Language Models (MLLMs), we find that current paradigms still suffer from an important yet remain underexplored issue : editing decoupling failure, where entity-related knowledge can be updated when the model is triggered by multimodal inputs (text--image query pairs), however, it often reverts to outdated pre-edit facts when the paired inputs are split into unimodal ones.

By Tingchao Fu, Wenkai Wang, Fanxiao Li, Huadong Zhang, Jinhong Zhang, Dayang Li, Yunyun Dong, Renyang Liu, Wei Zhou

llms multimodal

arXiv AI

Jul 29

CHARM: A Multimodal Graph Foundation Model with Hierarchical Context Modeling for Zero-Shot Transfer

arXiv:2607. 26023v1 Announce Type: new Abstract: Graph foundation models (GFMs) have emerged as a promising paradigm for transferring knowledge across graph domains and tasks.

By Ankang Yang, Jitao Zhao, Di Jin, Yuxiao Huang, Dongxiao He

llms fine-tuning multimodal

arXiv Machine Learning

Aug 4

Towards Effective Federated Multimodal Graph Learning via Navigating Multifaceted Heterogeneity

arXiv:2608. 00623v1 Announce Type: new Abstract: Multimodal-attributed graphs (MAGs), where nodes carry heterogeneous semantic content across multiple modalities while edges encode relational dependencies, have been widely adopted across diverse domains.

By Yinlin Zhu, Di Wu, Yi Zhang, Xunkai Li, Wang Luo, Wei-Jin Huang, Miao Hu, Guocong Quan

fine-tuning multimodal benchmarks

Related stories