Does the SORG Machine-learning Algorithm for Extremity Metastases Generalize to a Contemporary Cohort of Patients?...

Area of Science:

• Orthopedic oncology
• Machine learning in medicine
• Prognostic modeling

Background:

• Accurate survival prediction is vital for patients with osseous metastatic disease of the extremities.
• The Skeletal Oncology Research Group (SORG) developed a machine-learning algorithm (MLA) to predict 90-day and 1-year survival for surgically treated patients.
• Evolving oncology treatment regimens necessitate re-evaluating the SORG MLA's predictive accuracy over time.

Purpose of the Study:

• To determine if the SORG MLA accurately predicts 90-day and 1-year survival in a recent patient cohort (2016-2020).
• To validate the SORG MLA's performance in a contemporary patient population undergoing surgical treatment for metastatic long-bone lesions.

Main Methods:

• Temporal validation of the SORG MLA using data from 406 surgically treated patients (2016-2020).
• Analysis included perioperative laboratory values, tumor characteristics, and demographics.
• Model performance was assessed using the c-statistic (AUC), calibration plots, Brier scores, and decision curve analysis.

Main Results:

• The SORG MLA demonstrated decreased performance in the validation cohort compared to the development cohort.
• Area Under the Curve (AUC) values were 0.78 for 90-day and 0.75 for 1-year survival, indicating reasonable discrimination.
• Calibration analysis revealed overestimation of mortality risk, with Brier scores of 0.16 (90-day) and 0.22 (1-year), higher than the development cohort.

Conclusions:

• The SORG MLA showed decreased predictive performance on temporal validation, with a tendency to overestimate mortality risk.
• Temporal reassessment of MLA-driven survival calculators is essential due to potential performance decline over time.
• Clinicians should exercise caution and consider their experience when interpreting SORG MLA predictions, especially with evolving treatments like immunotherapy.