Evaluating the Utility and Limitations of Machine Learning Tumor Segmentation for Automated Longitudinal Response Assessment in Neuro-Oncology Treatment Response Classification

Machine learning segmentation has emerged in tumor assessment with high performance in volumetric evaluation of brain tumors. It is unclear, however, if such methods can be translated into full end-to-end clinical tools for assessing treatment response. This study aims to assess the ability and current limitations of machine learning for performing a fully automated longitudinal Response Assessment in Neuro-Oncology (RANO).

An nnU-Net model was trained for segmentation on 4162 MRIs of pre- and posttreatment brain tumors. A separate data set of 91 patients was preprocessed, including coregistration for longitudinal lesion tracking. The nnU-Net model was used to segment enhancing tumor on the longitudinal data set. After excluding patients classified as having progression due to T2/FLAIR per editorial preference changes or clinical criteria, 233 pair-wise MRI comparisons remained for 60 patients. Using auto-segmented tumor masks, comparisons were classified into progressive disease (PD), partial response (PR), stable disease (SD), and complete response (CR) according to the original RANO criteria based on 2D lesion size, measurable criteria, the appearance of new lesions, and changes over time. Model results were compared with expert ground truth and evaluated for accuracy in classifying PD and non-PD disease, per-class accuracy, and a variety of performance metrics. All discrepant cases were manually reviewed to classify the cause of the error.

The model was 77.7% (95% CI, 70.6%-84.2%) accurate in distinguishing PD and non-PD disease, with a sensitivity of 92.1% (86.2%-97.3%) and a specificity of 60.4% (43.4%-74.7%). The highest per-class accuracy was PD at 92.1% (86.2%-97.3%), with much lower performance for non-PD categories, and worst for PR at 14.3% (0.0%-38.5%). Patient-averaged accuracy was 66.9% (58.2%-75.3%). False-positive PD predictions were commonly due to evolving changes around the resection cavity, affecting the morphology and location of enhancement, accounting for 32.8% of SD classifications. Other causes of error included changing morphology but a similar volume of enhancement, missed lesions in less typical locations, and radiation necrosis.

Automated tumor segmentation is emerging in treatment response assessment, although highly trained segmentation models still have limitations in fully automated response assessment due to nuances such as postsurgical enhancement and comparing location and morphologic changes between time points.