Internal validation of risk models in lung resection surgery: bootstrap versus training-and-test sampling

Area of Science:

• Medical Statistics
• Surgical Outcomes Research

Background:

• Developing accurate mortality risk models is crucial for lung resection surgery.
• Traditional training-and-test methods may yield unreliable models due to dataset splitting.

Purpose of the Study:

• To compare the performance of a logistic regression and bootstrap analysis mortality model against traditional training-and-test models.
• To evaluate the reliability and discrimination of different risk model development strategies.

Main Methods:

• Generated 11 mortality models: 1 using logistic regression with bootstrap validation, 10 using traditional training-and-test splits.
• Evaluated model performance using c-statistics from 1000 bootstrap samples on an external dataset.

Main Results:

• The logistic regression and bootstrap model showed good discrimination (c-statistics >0.7 in 80% of samples).
• Only one of the 10 training-and-test models performed comparably; others showed poorer discrimination.

Conclusions:

• The traditional training-and-test method is unreliable for external validation in risk model building.
• Bootstrap analysis is superior for variable selection and model development in regression analysis.
• Bootstrap analysis is recommended for all future risk model development processes.