Evaluation of the dataset quality in gamma passing rate predictions using machine learning methods

Area of Science:

• Medical Physics
• Radiotherapy
• Machine Learning

Background:

• Machine learning (ML) models are used for gamma passing rate (GPR) predictions in radiotherapy plan verification.
• Existing methods often use heterogeneous datasets, leading to reduced model interpretability and performance.
• Dataset composition's impact on GPR classification models requires thorough investigation.

Purpose of the Study:

• To investigate the effect of dataset composition on GPR binary classification (pass/fail) using various ML models.
• To evaluate how heterogeneity factors (anatomical region, arcs, dose per fraction, treatment unit) influence model performance.
• To assess the impact of radiomic features on model accuracy.

Main Methods:

• Developed one reference and 24 customized datasets from 945 radiotherapy plans, varying heterogeneity factors.
• Extracted 309 features from plan parameters, modulation complexity, and radiomic analysis (leave-trajectory maps, 3D dose distributions, portal dosimetry images).
• Trained and evaluated Random Forest (RF), XG-boost, and Neural Network (NN) models, measuring performance via ROC-AUC.

Main Results:

• Radiomics features enhanced ROC-AUC by up to 13% (RF), 15% (XG-Boost), and 5% (NN) for reference models.
• Highly heterogeneous datasets yielded lower ROC-AUC values (e.g., RF: 0.72 ± 0.11) compared to less heterogeneous ones (e.g., RF: 0.88 ± 0.06).
• Top features correlated with treatment physics and GPR prediction, highlighting dataset influence.

Conclusions:

• Homogeneous datasets improve data generalization and ML model performance in radiotherapy.
• Dataset quality and composition are critical for reliable ML applications in radiation oncology.
• This analysis framework can guide dataset selection and model evaluation for future ML studies.