Understanding machine learning classifier decisions in automated radiotherapy quality assurance

Area of Science:

• Medical Physics
• Machine Learning
• Radiotherapy

Background:

• Radiotherapy treatment planning requires complex quality assurance (QA) to ensure patient safety and prevent errors.
• Machine learning (ML) classifiers offer potential for improving the efficiency and scope of traditional QA processes.
• A critical limitation of ML classifiers in QA is the lack of transparency in their decision-making.

Purpose of the Study:

• To develop and evaluate explanation methods for understanding the predictions of automated QA classifiers in radiotherapy.
• To assess the reliability and consistency of these explanation methods, specifically LIME and Shapley values.
• To determine the trustworthiness of ML classifiers for radiotherapy treatment plan QA.

Main Methods:

• Implemented Local Interpretable Model-Agnostic Explanations (LIME) and a novel team-based Shapley values framework.
• Applied these explanation methods to two types of automated QA classifiers: region of interest (ROI) segmentation/labeling and treatment plan acceptance.
• Tested the methods on datasets from prostate and breast radiotherapy treatment sites and introduced explanation consistency metrics.

Main Results:

• Both LIME and Shapley values provide visualizations for human expert assessment of classifier decisions.
• The team-based Shapley values approach demonstrated greater consistency compared to LIME.
• The evaluated QA classifiers were found to be moderately trustworthy and suitable for confirming expert decisions.

Conclusions:

• Interpretable machine learning approaches are crucial for evaluating and validating automated QA classifiers in radiotherapy.
• Explanation methods enhance the transparency and trustworthiness of ML-driven QA tools.
• Current automated QA classifiers can support, but should not replace, the human QA process in radiotherapy.