Artificial Intelligence for Prognostic Scores in Oncology: a Benchmarking Study.

Area of Science:

• Oncology
• Biostatistics
• Machine Learning

Background:

• Prognostic scores are crucial for oncology clinical decisions.
• Classical survival analysis using Cox regression is standard for developing prognostic scores.
• Advancements in analytical models prompt investigation into machine learning's potential.

Purpose of the Study:

• To benchmark complex machine-learning algorithms against classical survival analysis for prognostic scoring.
• To evaluate model performance across different feature set sizes.
• To assess generalizability of prognostic models using in-sample and out-of-sample validation.

Main Methods:

• Benchmarking study using two datasets: a large EHR-derived database and the OAK clinical trial dataset.
• Developed and compared prognostic models for overall survival in pan-cancer and non-small cell lung cancer populations.
• Evaluated Cox model-based ROPRO against eight machine-learning models (RSF, GB, DS, AE, SL) using C-index metric across feature sets of 27, 44, and 88 covariates.

Main Results:

• Machine-learning models (RSF, GB, DS, SL) demonstrated significantly better in-sample performance (higher C-index) than ROPRO on the EHR database.
• This superior performance of complex models did not translate to the out-of-sample OAK dataset.
• Increasing the number of prognostic covariates did not consistently improve model performance.

Conclusions:

• The enhanced performance of complex machine-learning models in cancer prognostic scoring may not generalize to external datasets.
• Future research could explore multimodal data integration to leverage the capabilities of advanced models.
• The study highlights the importance of rigorous external validation for prognostic models.