Fast2Vec, a modified model of FastText that enhances semantic analysis in topic evolution

Area of Science:

• Natural Language Processing
• Computational Linguistics
• Data Science

Background:

• Topic modeling methods like LDA and DTM extract topics but require manual interpretation, hindering the analysis of semantic topic evolution, especially with negations, synonyms, or rare terms.
• Neural network-based word embeddings (Word2vec, FastText) advance semantic understanding but have limitations: Word2Vec struggles with out-of-vocabulary (OOV) words, and FastText yields suboptimal embeddings for infrequent terms.

Purpose of the Study:

• Introduce Fast2Vec, a novel model combining Word2Vec's semantic capabilities with FastText's subword analysis for enhanced topic modeling.
• Evaluate Fast2Vec's performance on research abstracts and validate it using word similarity benchmarks.

Main Methods:

• Developed Fast2Vec by integrating Word2Vec and FastText.
• Evaluated the model using research abstracts from the Science and Technology Index (SINTA) journal database.
• Validated using twelve public word similarity benchmarks and assessed with Spearman and Pearson correlation coefficients.

Main Results:

• Fast2Vec outperformed or matched Word2Vec and FastText in semantic similarity tasks.
• Improved semantic similarity by 39.64% for OOV words compared to Word2Vec and 6.18% compared to FastText.
• Effectively categorized topics into four evolution patterns: diffusion, shifting, moderate fluctuations, and stability.

Conclusions:

• Fast2Vec provides a robust, generalizable word embedding framework for semantic-based topic modeling.
• Effectively addresses limitations in handling OOV terms and semantic variation.
• Demonstrates strong potential for broader natural language processing applications.