Double-blind comparison of survival analysis models using a bespoke web system

Area of Science:

• Biostatistics
• Machine Learning in Healthcare
• Survival Analysis

Background:

• Accurate prognostic modeling is crucial for patient stratification and treatment planning.
• Various statistical and machine learning models exist for survival prediction, each with potential biases.
• Objective comparison of these models on a common dataset is needed to understand their performance and limitations.

Purpose of the Study:

• To conduct a double-blind comparison of diverse linear and non-linear survival models.
• To evaluate model performance using a standardized dataset and methodology.
• To identify potential biases, such as optimism or pessimism, in different modeling approaches.

Main Methods:

• A common dataset was securely shared via the geoconda environment.
• Evaluated models included Cox, Log Normal, Partial Logistic Spline, Partial Logistic Artificial Neural Network, and Radial Basis Function Networks.
• Model discrimination was assessed using the area under the Receiver Operating Characteristic (ROC) curve, with graphical analysis against Kaplan-Meier estimates in TNM-classified survival groups.

Main Results:

• The Cox model demonstrated a tendency towards optimism.
• Partial Logistic Artificial Neural Networks exhibited slight pessimism.
• Model performance varied, highlighting differences in predictive accuracy and bias.

Conclusions:

• Different survival models exhibit distinct biases when applied to real-world data.
• Artificial Neural Networks, while complex, showed a tendency towards conservatism in this evaluation.
• Careful model selection and bias assessment are essential for reliable survival prediction in clinical settings.