Evaluation of an automatic article selection method for timelier updates of the Comet Core Outcome Set database

Area of Science:

• Bibliometrics
• Information Science
• Computational Biology

Background:

• Manually screening scientific literature for curated databases is labor-intensive.
• The Comet Core Outcome Set database relies on manual screening, leading to time-consuming updates.
• Efficient literature searching is crucial for researchers.

Purpose of the Study:

• To evaluate machine learning methods for reducing the workload of manual literature screening.
• To assess the effectiveness of a logistic regression model in automatically ranking candidate articles for the Comet database.
• To determine if machine learning can facilitate more timely updates of systematic review databases.

Main Methods:

• A machine learning approach using logistic regression was employed to rank candidate articles.
• Data from the original systematic review and four subsequent updates were used for model training and evaluation.
• Performance was assessed based on workload reduction and the rate of missed references.

Main Results:

• The study estimated a workload reduction of at least 75% using automated screening.
• The proposed machine learning method is expected to keep the number of missed references at approximately 2%.
• This trade-off was deemed acceptable for the systematic review process.

Conclusions:

• Machine learning offers a viable solution to reduce the manual workload in curating scientific literature databases.
• The evaluated logistic regression model provides an efficient method for prioritizing articles in systematic reviews.
• The successful application of this method is being implemented for future updates of the Comet database.