Empirical analyses and simulations showed that different machine and statistical learning methods had differing...

Area of Science:

• Computational statistics
• Biomedical informatics
• Machine learning in healthcare

Background:

• Machine learning (ML) is widely applied for clinical outcome prediction.
• Comparisons of ML methods often rely on empirical dataset analyses.
• Understanding method performance across diverse scenarios is crucial for reliable clinical prediction.

Purpose of the Study:

• To compare the performance of six distinct machine learning and statistical learning methods.
• To determine conditions under which ML methods outperform traditional statistical approaches.
• To evaluate predictive accuracy for continuous clinical outcomes using simulations.

Main Methods:

• Monte Carlo simulations were employed to generate data for derivation and validation samples.
• Six learning methods were evaluated: boosted trees, random forests, neural networks, lasso, ridge regression, and OLS regression.
• Simulations were informed by real-world data from acute myocardial infarction (AMI) and congestive heart failure (CHF) patient cohorts, focusing on systolic blood pressure prediction.

Main Results:

• Artificial neural networks generally exhibited lower predictive accuracy compared to the other five methods.
• Stochastic gradient boosting machines (boosted trees) and ordinary least squares (OLS) regression demonstrated robust performance across various simulated scenarios.
• Performance varied depending on the data-generating process, highlighting the importance of method selection.

Conclusions:

• Boosted trees and OLS regression are reliable methods for predicting continuous clinical outcomes like systolic blood pressure.
• Artificial neural networks may not be optimal for all clinical prediction tasks, especially when compared to simpler models in certain settings.
• Simulation studies provide valuable insights into the comparative performance of predictive models in healthcare.