Overlooked pitfalls in multi-class machine learning classification in radiation oncology and how to avoid them.

Area of Science:

• Medical Physics
• Machine Learning
• Radiation Oncology

Background:

• Machine learning classification in radiation oncology often focuses on binary outcomes.
• Multi-class classification, such as predicting treatment complication grades, has significant clinical importance.
• Existing statistical measures may be inadequate for these complex scenarios.

Purpose of the Study:

• To address statistical challenges in multi-class classification for radiation oncology.
• To evaluate the appropriateness of correlation coefficients as alternatives to AUC for multi-class problems.
• To investigate the reliability of surrogate biomarkers for clinical endpoints using Monte Carlo simulations.

Main Methods:

• Utilized Monte Carlo (MC) models to simulate multi-class classification scenarios.
• Analyzed the validity of Pearson and Spearman correlation coefficients for nominal and ordinal classes.
• Simulated a clinical endpoint (radiation toxicity, scale 0-5) and a noisy surrogate biomarker across multiple patient cohorts.

Main Results:

• Demonstrated that correlation coefficients can be misleading for multi-class classification, especially with nominal classes.
• Showed that Pearson or Spearman correlation coefficients provide incomplete or meaningless information for nominal data.
• MC experiments revealed that surrogate biomarkers can frequently fail to distinguish between patient cohorts with statistically significant differences in the true endpoint, particularly with increased noise.

Conclusions:

• Standard statistical measures require careful consideration in multi-class machine learning for radiation oncology.
• Correlation coefficients are not universally suitable replacements for AUC in multi-class settings.
• The use of surrogate biomarkers necessitates rigorous validation to avoid erroneous clinical conclusions, as they can mask true treatment effects.