Comparison of Classically and Machine Learning Generated Survival Prediction Models for Patients With Spinal...

Area of Science:

• Orthopedic Oncology
• Medical Informatics
• Biostatistics

Background:

• The Skeletal Oncology Research Group (SORG) developed classical algorithms (CA) and machine learning algorithms (MLA) for predicting skeletal-related events.
• Evaluating the generalizability and performance variability of these algorithms across different geographic regions is essential for clinical application.

Purpose of the Study:

• To compare the discriminatory ability of SORG CA and SORG MLA using a meta-analysis.
• To test if SORG CA exhibits less performance variability than SORG MLA in non-American validation cohorts, hypothesizing that its exclusion of regional variables like BMI contributes to this.

Main Methods:

• A systematic review and meta-analysis were conducted to pool the discriminatory abilities (Area Under the Curve - AUC) of SORG CA and SORG MLA.
• Logit transformation of AUCs was used for analysis, with direct comparisons of logit(AUC) values.
• Subgroup analysis was performed to compare algorithm performance between American and non-American cohorts.

Main Results:

• Pooled logit(AUC) values indicated varying performance for both SORG CA and SORG MLA at 90-day and 1-year intervals.
• All algorithms demonstrated superior performance in the United States compared to Taiwan (P < .001).
• SORG CA's performance was less affected by non-American cohorts compared to SORG MLA.

Conclusions:

• The findings suggest that SORG CA is more robust across different regions than SORG MLA.
• The study highlights the importance of including region-specific variables in predictive models to enhance their generalizability to diverse populations.