DMutDE: Dual-View Mutual Distillation Framework for Knowledge Graph Embeddings.

Area of Science:

• Artificial Intelligence
• Data Science
• Machine Learning

Background:

• Knowledge Graphs (KGs) and Knowledge Graph Embeddings (KGE) are crucial for data representation and reasoning.
• Existing KGE models face challenges with spatial complexity, storage limitations, and reasoning efficiency.
• Current knowledge distillation (KD) methods rely on large teacher models, which are costly and resource-intensive.

Purpose of the Study:

• To develop a KGE enhancement framework that does not require large teacher models.
• To improve the performance and generalization of lightweight KGE models.
• To address the limitations of resource-constrained scenarios in KGE applications.

Main Methods:

• Propose the Dual-View Mutual Distillation Framework for Knowledge Graph Embeddings (DMutDE).
• Utilize mutual learning for peer-to-peer distillation between KGE models with different architectures.
• Introduce a soft-label fusion (SLF) module for noise filtering and response knowledge transfer.
• Implement an entity embedding distillation (EED) module for distilling structural features.

Main Results:

• The DMutDE framework achieves state-of-the-art results on standard open-source benchmarks.
• Demonstrates improved performance and generalization of student KGE models.
• Effectively enhances KGE models without the need for large, high-performance teacher models.

Conclusions:

• DMutDE offers an effective solution for enhancing KGE models in resource-constrained environments.
• The framework successfully integrates dual-view knowledge through mutual distillation.
• Provides a viable alternative to traditional KD approaches for KGE, reducing computational overhead.