Learning linear transformations between counting-based and prediction-based word embeddings.

Area of Science:

• Natural Language Processing
• Machine Learning
• Computational Linguistics

Background:

• Prediction-based word embedding methods are gaining traction.
• The relationship between prediction-based and counting-based word embeddings is not well understood.
• It is unclear if these different embedding spaces can be transformed into one another.

Purpose of the Study:

• To investigate the relationship between prediction-based and counting-based word embeddings.
• To propose a method for learning linear transformations between word embedding spaces.
• To demonstrate the ability to predict word embeddings for unseen words.

Main Methods:

• Developed an efficient algorithm to learn a linear transformation between two sets of word embeddings.
• Applied the learned transformation to predict embeddings for novel words.

Main Results:

• Successfully learned a linear transformation between different word embedding spaces.
• Empirically demonstrated the prediction of distributed word embeddings for novel, unseen words using the learned transformation.

Conclusions:

• Prediction-based and counting-based word embeddings are linearly transformable.
• The proposed method enables predicting embeddings for new words, bridging the gap between different embedding techniques.