A Performance Comparison of Unsupervised Techniques for Event Detection from Oscar Tweets

Area of Science:

• Computational Social Science
• Data Mining
• Natural Language Processing

Background:

• Social media is a vital source for real-time information and event detection.
• Existing research often focuses on specific datasets, limiting the evaluation of topic detection methods.
• There is a need for diverse datasets to assess the generic performance of these techniques.

Purpose of the Study:

• To introduce a novel dataset of 2,160,738 English tweets related to the Oscars 2018 event.
• To compare the performance of unsupervised topic detection and tracking methods on this real-world dataset.
• To evaluate the generic applicability of methods like Soft Frequent Pattern Mining (SFPM), Singular Value Decomposition and K-Means (K-SVD), Feature-Pivot (Feat-p), Document-Pivot (Doc-p), and Latent Dirichlet Allocation (LDA).

Main Methods:

• Collection of a large-scale Twitter dataset focusing on the Oscars 2018.
• Application of five unsupervised learning algorithms: SFPM, K-SVD, Feat-p, Doc-p, and LDA.
• Evaluation using metrics: keyword precision (K-Prec), keyword recall (K-Rec), and topic recall (T-Rec).

Main Results:

• SFPM achieved the highest K-Prec, K-Rec, and T-Rec, although performance decreased with increased clusters.
• Feat-p demonstrated the lowest performance across all evaluated metrics.
• All tested unsupervised methods proved to be generic and capable of producing meaningful clusters from the social media data.

Conclusions:

• The Oscars 2018 dataset provides a valuable resource for evaluating topic detection methods.
• SFPM is a highly effective method for event detection in social media.
• Unsupervised methods show promise for generic event and topic analysis on diverse datasets.