Lifelong knowledge graph embedding via diffusion model

Area of Science:

• Artificial Intelligence
• Machine Learning
• Data Science

Background:

• Lifelong knowledge graph embedding (KGE) methods are crucial for continuous learning in large, evolving knowledge graphs.
• Existing KGE methods struggle with catastrophic forgetting and inefficient learning due to embedding space drift.
• The need for methods that retain old knowledge while learning new facts is growing.

Purpose of the Study:

• To propose a novel lifelong KGE framework addressing embedding space drift and catastrophic forgetting.
• To enable continuous learning by unifying the learning of new facts and preservation of old facts.
• To enhance knowledge retention and transfer while reducing training costs.

Main Methods:

• A diffusion-based embedding method to capture contextual variations and generate transferable embeddings.
• A reconstruction and generation strategy using contrastive learning to manage embedding space drift and learning efficiency.
• Distribution regularization to prevent catastrophic forgetting and stabilize embedding distributions.

Main Results:

• The proposed framework demonstrated superior performance compared to existing lifelong KGE methods.
• Extensive experiments on seven benchmark datasets confirmed the framework's effectiveness across various incremental learning scenarios.
• The methods successfully addressed embedding space drift and improved learning efficiency.

Conclusions:

• The novel lifelong KGE framework effectively mitigates catastrophic forgetting and enhances learning efficiency.
• The diffusion-based embeddings and contrastive learning strategies are key to preserving knowledge and adapting to new information.
• This work provides a significant advancement in lifelong learning for knowledge graphs.