An Effective Meaningful Way to Evaluate Survival Models

Area of Science:

• Biostatistics
• Machine Learning
• Data Science

Background:

• Evaluating survival prediction models often uses Mean Absolute Error (MAE).
• Right-censored data in test sets complicates accurate MAE calculation.
• Existing metrics may not reliably assess model performance with censored individuals.

Purpose of the Study:

• To explore and propose effective metrics for estimating MAE in survival datasets with right-censored individuals.
• To introduce a novel method for generating realistic semi-synthetic survival datasets for metric evaluation.

Main Methods:

• Investigated various metrics for estimating MAE on survival data containing censored individuals.
• Developed a novel approach for creating semi-synthetic survival datasets.
• Compared the performance of different MAE estimation metrics using the generated datasets.

Main Results:

• The proposed metric, MAE using pseudo-observations, accurately ranks survival model performance.
• This novel metric closely approximates the true MAE and outperforms several alternative methods.
• Semi-synthetic datasets proved effective for evaluating survival metric performance.

Conclusions:

• MAE using pseudo-observations is a reliable metric for evaluating survival prediction models with censored data.
• The proposed method for generating semi-synthetic data aids in robust metric assessment.
• This work provides a more accurate way to assess survival model performance in the presence of censoring.