Experiences with the Twitter Health Surveillance (THS) System

Area of Science:

• Computer Science
• Data Science
• Public Health Informatics

Background:

• Social media platforms are crucial for understanding public opinion on various topics.
• Analyzing the high volume and velocity of social media data presents significant big data challenges.
• Existing solutions often involve complex, difficult-to-maintain collections of disparate software tools.

Purpose of the Study:

• To present the Twitter Health Surveillance (THS) application framework.
• To provide a platform for monitoring and processing social media streams, specifically tweets.
• To enable end-users to combine built-in functionalities with custom processing logic.

Main Methods:

• Implementation of the THS framework utilizing the Apache Hadoop Ecosystem.
• Development of a system to capture and analyze a continuous stream of tweets.
• Integration of built-in processing capabilities with user-defined functions for flexible analysis.

Main Results:

• Successful development of the Twitter Health Surveillance (THS) prototype.
• Demonstration of THS's capability to handle and process tweet streams effectively.
• Identification of practical challenges and lessons learned during the development process.

Conclusions:

• The THS framework offers a robust solution for social media surveillance and public opinion analysis.
• Leveraging the Apache Hadoop Ecosystem provides a scalable foundation for big data processing.
• The developed framework simplifies the creation and maintenance of social media analytics applications.