Screening PubMed abstracts: is class imbalance always a challenge to machine learning?

Area of Science:

• Bibliometrics
• Medical Informatics
• Computational Biology

Background:

• Increasing medical literature necessitates efficient systematic review citation screening.
• Researchers face a growing workload due to vast online textual data.

Purpose of the Study:

• To identify the optimal machine learning strategy for screening PubMed articles for systematic reviews.
• To evaluate data preprocessing techniques for class imbalance in this context.

Main Methods:

• Trained four binary text classifiers (SVM, k-NN, Random Forest, Elastic-Net).
• Applied four class imbalance techniques: random undersampling and oversampling (50:50, 35:65 ratios) and none.
• Used 14 systematic reviews' data; calculated delta AUC (difference in AUC-ROC with/without preprocessing) per classifier.
• Employed meta-analytic fixed-effect models to pool delta AUCs.

Main Results:

• Machine learning classifiers (except k-NN) achieved >90% AUC-ROC without preprocessing.
• Best performance improvements were seen with random oversampling (50:50) and random undersampling (35:65).

Conclusions:

• Resampling techniques offered slight performance enhancements for machine learning classifiers.
• Random undersampling (35:65) is recommended for computational efficiency.