Text embedding techniques for efficient clustering of twitter data

Area of Science:

• Natural Language Processing (NLP)
• Machine Learning
• Data Science

Background:

• The internet contains vast amounts of unstructured text data from sources like blogs and social media.
• Understanding this data is crucial for applications such as sentiment analysis and event detection.
• Natural Language Processing (NLP) provides methods to analyze and derive insights from textual information.

Purpose of the Study:

• To evaluate various word embedding techniques for analyzing tweets from popular news channels.
• To cluster the resulting word vectors using the K-means algorithm.
• To identify the most accurate word embedding method for this task.

Main Methods:

• Application of multiple word embedding techniques on a dataset of news channel tweets.
• Clustering of the generated word vectors using the K-means algorithm.
• Comparative analysis of the accuracy of different word embedding methods.

Main Results:

• Bidirectional Encoder Representations from Transformers (BERT) demonstrated superior performance.
• BERT, when combined with K-means clustering, achieved the highest accuracy rate.
• The study quantifies text data using word embeddings for better analysis.

Conclusions:

• Word embedding techniques, particularly BERT, are effective for analyzing and clustering tweet data.
• The combination of BERT and K-means clustering offers a highly accurate approach for NLP tasks.
• This method can be applied to understand insights from large volumes of online text.