Prediction of brain metastasis progression after stereotactic radiosurgery: sensitivity to changing the definition of...

Machine learning (ML) has been used to predict tumor progression post-stereotactic radiosurgery (SRS) based on pre-treatment MRI for brain metastasis (BM) patients, but there is variability in the definition of what constitutes progression. We aim to measure the magnitude of the change of performance of an ML model predicting post-SRS progression when various definitions of progression were used.

We collected pre- and post-SRS contrast-enhanced T1-weighted MRI scans from 62 BM patients ( $n=115$ BMs). We trained a random decision forest model using radiomic features extracted from pre-SRS scans to predict progression versus non-progression for each BM. We varied the definition of progression by changing (1) the follow-up period ( $<9$ , $<12$ , $<15$ , $<18$ , or $<24$ months); (2) the size change metric denoting progression ( $\ge 10\%$ , $\ge 15\%$ , $\ge 20\%$ , or $\ge 25\%$ in volume) or response assessment in neuro-oncology BM diameter ( $\ge 20\%$ ); and (3) whether BMs with treatment-related size changes (TRSCs) (pseudo-progression and/or radiation-necrosis) were labeled as progression. We measured performance using the area under the receiver operating characteristic curve (AUC).

When we varied the follow-up period, size change metric, and TRSC labeling, the AUCs had ranges of 0.06 (0.69 to 0.75), 0.06 (0.69 to 0.75), and 0.08 (0.69 to 0.77), respectively. Radiomic feature importance remained similar.

Variability in the definition of BM progression has a measurable impact on the performance of an MRI radiomic-based ML model predicting post-SRS progression. A consistent, clinically relevant definition of post-SRS progression across studies would enable robust comparison of proposed ML systems, thereby accelerating progress in this field.