Effective Training Data Extraction Method to Improve Influenza Outbreak Prediction from Online News Articles: Deep...

Area of Science:

• Computational epidemiology
• Machine learning for public health

Background:

• Influenza causes millions of illnesses and hundreds of thousands of deaths globally each year.
• Current surveillance systems face delays, hindering timely warnings.
• Existing internet data-driven prediction methods struggle with subjective data extraction and capturing latent features.

Purpose of the Study:

• To propose an effective method for extracting training data that reflects hidden features.
• To improve influenza prediction performance by filtering and selecting relevant keywords before prediction.

Main Methods:

• Enhanced word embeddings by selecting influenza-related keywords.
• Sorted keywords using Pearson correlation coefficient to retain high-correlation tokens.
• Employed a long short-term memory (LSTM) model for influenza outbreak prediction.
• Assessed model performance using various word embedding techniques.

Main Results:

• The proposed sorting process improved prediction accuracy from 0.8705 to 0.8868.
• Achieved a 12.6% increase in prediction accuracy with fewer keywords (20.6 vs. 50.2).
• Demonstrated superior performance compared to methods without the sorting stage.

Conclusions:

• The sorting stage enhances feature extraction, acting as a knowledge base for the prediction model.
• The proposed method significantly improves influenza prediction accuracy and efficiency.
• Outperformed current approaches relying on flat data extraction prior to prediction.