A link prediction method for multi-modal knowledge graphs based on Adaptive Fusion and Modality Information...

Area of Science:

• Artificial Intelligence
• Data Science
• Knowledge Representation

Background:

• Multi-modal knowledge graphs (MMKGs) enhance traditional knowledge graphs with diverse information, showing promise in reasoning tasks.
• Existing MMKGs struggle with link prediction due to complex, diverse, and imbalanced multi-modal data, hindering effective information fusion and enhancement.
• Current fusion methods often use simple concatenation or weighting, failing to capture deep semantic interactions and performing poorly with noisy or missing data.

Purpose of the Study:

• To propose a novel framework, Adaptive Fusion and Modality Information Enhancement (AFME), to address the challenges in multi-modal knowledge graph link prediction.
• To develop efficient adaptive fusion and enhancement mechanisms for multi-modal information within knowledge graphs.
• To improve the utilization of multi-modal features and enhance knowledge reasoning capabilities in complex scenarios.

Main Methods:

• The AFME framework comprises a Modal Information Fusion (MoIFu) module and a Modal Information Enhancement (MoIEn) module.
• Introduced a relationship-driven denoising mechanism and dynamic weight allocation for efficient adaptive fusion.
• Employed a generative adversarial network (GAN) for global guidance and a multi-layer self-attention mechanism for intra- and inter-modal feature optimization.

Main Results:

• The AFME framework demonstrated significant improvements in multi-modal feature utilization across multiple benchmark datasets.
• Experimental results validated the framework's efficiency and superiority in complex multi-modal knowledge graph scenarios.
• The proposed methods effectively addressed issues of modal noise and missing information, enhancing link prediction accuracy.

Conclusions:

• The AFME framework offers a robust solution for challenges in multi-modal knowledge graph completion reasoning.
• The adaptive fusion and enhancement strategies significantly boost knowledge reasoning performance.
• AFME represents a substantial advancement in leveraging diverse information for more capable knowledge graphs.