Protocol for a systematic review on the methodological and reporting quality of prediction model studies using...

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Clinical Prediction Models

Background:

• Prediction model studies are numerous but often have suboptimal methodological and reporting quality.
• The increasing use of Artificial Intelligence (AI) and machine learning (ML) in developing prediction models presents new challenges, often referred to as 'black boxes'.
• There is a need to evaluate the quality of AI/ML-based prediction models due to their growing prevalence.

Purpose of the Study:

• To comprehensively evaluate the reporting quality, methodological conduct, and risk of bias in prediction model studies utilizing AI or ML techniques.
• To assess adherence to the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) guidelines.
• To evaluate the risk of bias using the Prediction model Risk Of Bias ASsessment Tool (PROBAST).

Main Methods:

• A systematic search of PubMed for studies published between January 2018 and December 2019 on AI/ML-based prediction models.
• Inclusion of studies predicting patient-related outcomes across all medical fields, using any study design or data source.
• Screening and data extraction by two independent reviewers, with primary outcomes focusing on TRIPOD adherence and PROBAST risk of bias assessment.

Main Results:

• A narrative synthesis of findings will be conducted.
• Results will be stratified by study type, medical field, and prevalent AI/ML methods.
• The findings will inform potential extensions or updates to TRIPOD and PROBAST guidelines.

Conclusions:

• The review will provide insights into the quality of AI/ML-based prediction models.
• It aims to enhance the transparency and reliability of these models in clinical practice.
• Recommendations for improving reporting and methodological standards for AI/ML prediction models will be developed.