Machine learning-based COVID-19 prognostic models lag behind in reporting quality: findings from a TRIPOD/TRIPOD + AI...

Area of Science:

• Medical Informatics
• Biostatistics
• Artificial Intelligence in Medicine

Background:

• Established reporting standards for COVID-19 prognostic models are frequently unmet.
• The Transparent Reporting of a multivariable Prediction model for Individual Diagnosis or Prognosis (TRIPOD) checklist and its 2024 AI extension (TRIPOD+AI) offer frameworks for quality assessment.
• This study compares reporting completeness between conventional and machine learning (ML) prognostic models for COVID-19.

Purpose of the Study:

• To evaluate and compare the reporting completeness of COVID-19 prognostic models developed using conventional statistical methods versus machine learning algorithms.
• To identify specific areas of deficiency in the reporting of these models.

Main Methods:

• A systematic literature search was conducted in MEDLINE, Epistemonikos.org, and Scopus up to July 31, 2024.
• Studies reporting development and validation of COVID-19 prognostic models were included.
• Conventional models were assessed using TRIPOD; ML models were assessed using TRIPOD+AI, with data extraction following checklist items.

Main Results:

• A total of 53 studies (71 models) were analyzed, showing low adherence to both TRIPOD and TRIPOD+AI guidelines.
• Machine learning models exhibited significantly poorer reporting completeness (28.4%) compared to conventional models (38.1%).
• No study fully adhered to abstract reporting, and sample size calculations were universally unreported; methods and results reporting were poor across all studies.

Conclusions:

• The lower adherence in ML studies may be attributed to the recent publication of TRIPOD+AI.
• Both conventional and ML-based COVID-19 prediction models demonstrate insufficient reporting, with critical gaps in model description and performance.
• Enhanced compliance with reporting guidelines is essential for improving the clarity, reproducibility, and clinical utility of prognostic prediction models.