Limited generalizability and high risk of bias in multivariable models predicting conversion risk from mild cognitive...

Area of Science:

• Neuroscience
• Gerontology
• Medical Informatics

Background:

• Mild cognitive impairment (MCI) is a transitional stage between normal aging and dementia.
• Accurate prediction of MCI to dementia conversion is crucial for timely intervention and management.
• Existing prediction models vary in their development, validation, and data sources.

Purpose of the Study:

• To systematically review and summarize multi-source prediction models for MCI to dementia conversion.
• To assess the predictors, performance, risk of bias, and generalizability of these models.
• To identify gaps and provide recommendations for future research in MCI prediction.

Main Methods:

• Systematic literature search of PubMed and Embase databases up to January 18, 2024.
• Inclusion of model development and validation studies for MCI to dementia conversion.
• Assessment of models based on predictors, predictive performance (e.g., AUC, accuracy), risk of bias, and generalizability.

Main Results:

• 62 studies were included, comprising 41 machine learning, 11 regression models, and 5 disease state indexes.
• Common predictors included magnetic resonance imaging (MRI) and cognitive scores; models used 2 to 60 predictors.
• Models demonstrated reasonable predictive capabilities (AUC: 0.58-0.98, accuracy: 66.1-96.3%), but most studies had high risk of bias and lacked external validation (47 studies).

Conclusions:

• No current prediction model for MCI to dementia conversion is highly valid due to limited generalizability and high risk of bias.
• While models show promising performance, significant limitations in external validation and generalizability persist.
• Future efforts must prioritize external validation, generalizability, and clinical applicability for robust MCI to dementia prediction models.