Fine-Tuning Word Embeddings for Hierarchical Representation of Data Using a Corpus and a Knowledge Base for Various...

Area of Science:

• Natural Language Processing
• Computational Linguistics
• Artificial Intelligence

Background:

• Word embedding models capture semantic relationships but often fail to explicitly represent hierarchical structures.
• Existing methods lack explicit encoding of hierarchical information present in knowledge bases.
• There is a need for word embeddings that effectively integrate lexical hierarchies.

Purpose of the Study:

• To propose a novel method for learning hierarchical word embeddings (HWEs).
• To explicitly encode the hierarchical structure of knowledge bases (KBs) into a vector space.
• To improve the representation of hierarchical information in natural language processing tasks.

Main Methods:

• Developed a method to learn HWEs by considering hypernym relations from KBs.
• Incorporated contextual information from text corpora into the embedding learning process.
• Designed a specific ordering for learning embeddings to capture hierarchy.

Main Results:

• The proposed HWEs successfully encode hierarchical information.
• Experimental results demonstrate effectiveness across supervised and unsupervised hypernymy detection, graded lexical entailment, and hierarchical path prediction.
• The method outperforms existing approaches for specialized and non-specialized hierarchical word embeddings on multiple benchmarks.

Conclusions:

• The proposed method effectively learns hierarchical word embeddings by combining KB hypernyms and textual context.
• HWEs provide a superior representation for tasks requiring explicit hierarchical understanding.
• This approach advances the field of word embeddings for knowledge representation and NLP applications.