Evaluation of RMES, an Automated Software Tool Utilizing AI, for Literature Screening with Reference to Published...

Area of Science:

• Medical Informatics
• Evidence-Based Medicine
• Bibliometrics

Background:

• Systematic reviews and meta-analyses are crucial for evidence-based medicine.
• Information retrieval and literature screening are time-consuming tasks in systematic reviews.
• Rapid Medical Evidence Synthesis (RMES) is a software tool designed to streamline these processes.

Purpose of the Study:

• To evaluate the accuracy of RMES for literature screening in systematic reviews.
• To assess RMES performance against published systematic review data.

Main Methods:

• RMES was used to screen titles and abstracts of PubMed-indexed articles from 12 systematic reviews.
• Four filtering approaches were applied: study type, study type + disease, study type + intervention, and study type + disease + intervention.
• Accuracy was determined by comparing RMES-identified articles to those included in the original systematic reviews.

Main Results:

• Article counts analyzed by RMES varied from 46 to 5612 across the 12 reviews.
• The median accuracy rates for the four filters ranged from 58.6% to 80.9%.
• RMES demonstrated variable but notable performance in correctly identifying relevant articles.

Conclusions:

• RMES shows good performance and accuracy for the initial screening of articles in systematic reviews.
• The software has the potential to substantially decrease the manual effort required for literature screening.
• RMES can aid researchers in evidence-based medicine by improving efficiency.