Improving Systematic Review Updates With Natural Language Processing Through Abstract Component Classification and...

Area of Science:

• Bibliometrics
• Information Science
• Computational Linguistics

Background:

• Systematic review updates face challenges due to extensive article screening workloads.
• Current natural language processing (NLP) screening models often treat abstracts uniformly, limiting performance.
• Selective training on specific abstract components is hypothesized to improve model efficacy.

Purpose of the Study:

• To evaluate a novel screening model that utilizes specific abstract components for improved performance.
• To develop an automated systematic review update model employing an abstract component classifier.

Main Methods:

• Developed screening models using component-composition datasets derived from manually classified abstract components (Title, Introduction, Methods, Results, Conclusion).
• Compared performance of models using Bidirectional Encoder Representations from Transformer (BERT), BioLinkBERT, and BioM-ELECTRA pre-trained models.
• Created an Abstract Component Classifier Model to automate component selection and developed models using these automatically classified datasets.

Main Results:

• Some models trained on specific components outperformed those trained on entire abstracts across all tested pre-trained models.
• Models using automatically classified components also surpassed full-abstract models in performance.
• Achieved an 88.6% reduction in manual screening workload with high recall (0.93).

Conclusions:

• Component selection from titles and abstracts demonstrably enhances screening model performance.
• This method substantially reduces manual screening workload for systematic review updates.
• Further validation across diverse systematic review domains is recommended.