Twitter mining using semi-supervised classification for relevance filtering in syndromic surveillance.

Area of Science:

• Public Health
• Computer Science
• Social Media Analysis

Background:

• Traditional syndromic surveillance systems face limitations in detecting individuals who do not seek direct healthcare.
• Twitter data presents a potential supplementary source for real-time public health monitoring.

Purpose of the Study:

• To evaluate Twitter data's utility in augmenting syndromic surveillance for asthma/difficulty breathing.
• To develop and assess semi-supervised classification methods for identifying relevant symptomatic tweets.

Main Methods:

• Collected Twitter data using the streaming API and pre-processed the information.
• Investigated semi-supervised text classification techniques for relevance filtering.
• Explored the impact of emojis and tweet tone on classification performance.

Main Results:

• Semi-supervised classification, particularly with negative or laughter emojis and n-gram models, effectively filtered symptomatic tweets.
• The proposed methods preserved more relevant tweets compared to traditional approaches, beneficial for weak signals.
• A significant correlation (r = 0.414, p = 0.0004) was found between the Twitter-derived signal and health consultation data.

Conclusions:

• Twitter data, when analyzed with appropriate semi-supervised methods, can serve as a valuable tool for syndromic surveillance.
• This approach enhances the understanding of symptomatic individuals not accessing traditional healthcare.
• The study highlights the potential of social media data in public health early detection efforts.