Assessing the effects of hyperparameters on knowledge graph embedding quality.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Data Science

Background:

• Knowledge graph embeddings are vital for tasks like link prediction and node classification.
• Current embedding methods require significant computational resources due to hyperparameter optimization.
• Hyperparameter tuning involves extensive sampling and testing, leading to high costs.

Purpose of the Study:

• To reduce the computational cost of knowledge graph embedding by identifying and prioritizing important hyperparameters.
• To investigate the impact of hyperparameter tuning on embedding quality using sensitivity analysis.
• To address data leakage issues in the UMLS knowledge graph and propose a robust variant.

Main Methods:

• Sobol sensitivity analysis was employed to assess hyperparameter influence on embedding quality variance.
• Thousands of embedding trials were conducted with varying hyperparameter configurations.
• Regression models were used to calculate Sobol indices for each hyperparameter.
• The UMLS knowledge graph was analyzed for inverse relations causing data leakage.

Main Results:

• Significant variability in hyperparameter sensitivity was observed across different knowledge graph datasets.
• Dataset characteristics were identified as a probable cause for these inconsistencies.
• Several relations in the UMLS knowledge graph were found to contribute to data leakage.
• A leakage-robust variant, UMLS-43, was derived from the original UMLS graph.

Conclusions:

• Prioritizing key hyperparameters can significantly reduce the computational burden of knowledge graph embedding.
• Understanding dataset-specific hyperparameter importance is crucial for efficient embedding.
• Addressing data leakage is essential for reliable knowledge graph analysis, as demonstrated by the UMLS-43 variant.