MERGE: A Modal Equilibrium Relational Graph Framework for Multi-Modal Knowledge Graph Completion

Area of Science:

• Artificial Intelligence
• Knowledge Representation and Reasoning

Background:

• Multi-modal knowledge graph completion (MMKGC) is vital for cross-modal learning but struggles with missing modal data.
• Existing methods often overlook the negative impact of incomplete multi-modal information.

Purpose of the Study:

• To propose an innovative framework, MERGE, that addresses challenges in MMKGC caused by missing modal information.
• To enhance the accuracy and robustness of multi-modal knowledge graph completion.

Main Methods:

• Developed a Modal Equilibrium Relational Graph framework (MERGE) using three modal-specific directed relational graph attention networks.
• Employed low-rank tensor decomposition for multi-modal feature fusion at structural and semantic levels.
• Introduced an interpolation re-ranking strategy to dynamically adjust modality importance during inference.

Main Results:

• MERGE implicitly represents missing modal information by aggregating embeddings from neighboring nodes.
• The fusion approach enhances interpretability by leveraging inherent knowledge graph structures.
• Experimental validation on four datasets confirmed MERGE's effectiveness and robustness in MMKGC.

Conclusions:

• MERGE provides an effective solution for multi-modal knowledge graph completion, particularly in scenarios with missing modal data.
• The framework demonstrates significant improvements in handling cross-modal learning and reasoning tasks.