Jointly learning word embeddings using a corpus and a knowledge base.

Area of Science:

• Natural Language Processing
• Computational Linguistics
• Artificial Intelligence

Background:

• Traditional word embedding methods rely solely on text corpora, often overlooking valuable semantic relationships.
• Manually curated knowledge bases (KBs) like ontologies capture rich word relationships but are not always integrated with corpus-based methods.
• Existing approaches struggle to fully leverage the complementary strengths of both textual data and structured knowledge.

Purpose of the Study:

• To develop a joint word representation learning method that effectively combines information from text corpora and knowledge bases.
• To enhance the accuracy and semantic richness of word embeddings by incorporating relational knowledge.
• To introduce novel techniques for dynamically expanding knowledge bases to improve embedding generation.

Main Methods:

• A joint learning framework that uses corpus co-occurrence statistics and relational constraints derived from KBs.
• Development of Nearest Neighbour Expansion (NNE) and Hedged Nearest Neighbour Expansion (HNE) for dynamic KB augmentation.
• An objective function defined by corpus data, constrained by KB-derived relationships.

Main Results:

• The proposed method significantly improves the accuracy of learned word embeddings compared to corpus-only baselines.
• Experimental results demonstrate superior performance on benchmark tasks, including semantic similarity prediction and word analogy detection.
• The novel NNE and HNE approaches effectively expand the KB, providing more guidance for the optimization process.

Conclusions:

• Integrating knowledge from both text corpora and KBs leads to demonstrably better word embeddings.
• The proposed joint learning method offers a powerful approach for enhancing natural language understanding.
• Dynamic KB expansion techniques further refine embedding quality, outperforming existing hybrid methods.