Towards identifying drug side effects from social media using active learning and crowd sourcing

Area of Science:

• Pharmacovigilance
• Computational Linguistics
• Data Science

Background:

• Social media platforms like Twitter are rich, yet underutilized, sources for real-world drug side effect information.
• Challenges in analyzing this data include sparse labeled training sets and the unreliability of automatic labeling methods.
• Crowdsourcing offers improved label accuracy but incurs significant costs.

Purpose of the Study:

• To develop and evaluate a semi-supervised active learning approach for efficiently and accurately labeling drug side effects from Twitter data.
• To reduce the manual labeling effort required for pharmacovigilance using noisy social media data.

Main Methods:

• Data extraction from Twitter via the public API.
• Two-stage labeling of tweets for drugs and side effects using Amazon Mechanical Turk and a semi-supervised active learning strategy.
• Comparison against a one-stage workflow and a non-active baseline.

Main Results:

• The proposed method significantly improved the F-measure for side effect discovery from 53% to 67%.
• Active learning demonstrated effectiveness in reducing overall labeling costs compared to a non-active baseline.
• The two-stage labeling process proved more effective than a one-stage workflow.

Conclusions:

• Semi-supervised active learning is a cost-effective and accurate method for identifying drug side effects from social media.
• This approach enhances pharmacovigilance by efficiently utilizing noisy, large-scale datasets like Twitter.
• Published code and data will facilitate further research in this area.