Comparing the performance of statistical, machine learning, and deep learning algorithms to predict time-to-event: A...

Area of Science:

• Computational neuroscience
• Geriatric medicine
• Biostatistics

Background:

• Mild Cognitive Impairment (MCI) detection is vital for dementia intervention.
• Predictive algorithms for time-to-event data in MCI are numerous but their comparative performance is unclear.
• The accuracy of algorithms with fewer training weights is often questioned.

Purpose of the Study:

• To compare the predictive accuracy of statistical (CoxPH), machine learning (RSF), and deep learning (DeepSurv) algorithms for time to MCI conversion.
• To investigate the impact of training weights and unobserved heterogeneity on algorithm performance.
• To inform the development of AI for time-to-event predictions in cognitive decline.

Main Methods:

• Simulated a dataset based on the Alzheimer NACC dataset to compare CoxPH, RSF, and DeepSurv models.
• Evaluated algorithm performance across different sample sizes and scenarios, including unobserved heterogeneity.
• Analyzed accuracy based on the number of training weights and model complexity.

Main Results:

• The Cox proportional hazards model (CoxPH) demonstrated strong performance across all simulated scenarios.
• DeepSurv achieved comparable accuracy (73.1%) to CoxPH (73%) in larger sample sizes (n=6,000).
• Accounting for heterogeneity in CoxPH yielded accuracy comparable to DeepSurv and RSF, debunking the notion of deep learning superiority.

Conclusions:

• Statistical models with fewer training weights can be as accurate as complex deep learning models for MCI prediction.
• Ignoring heterogeneity can lead to misinterpretations of algorithm performance.
• This study advocates for a principled approach to comparing AI algorithms for time-to-event predictions, favoring explainable models.