Is the Validity of Logistic Regression Models Developed with a National Hospital Database Inferior to Models...

Area of Science:

• Medical Informatics
• Health Services Research
• Clinical Epidemiology

Background:

• National hospital databases often lack detailed prognostic factors.
• Evaluating the performance of predictive models across different data sources is crucial for clinical utility.

Purpose of the Study:

• To compare the performance of prognostic models developed using a specialized clinical database (Epithor) versus a national French hospital database.
• To assess the impact of data source limitations on model accuracy and calibration.

Main Methods:

• Two datasets (Epithor and French hospital database) were used, with 70% for training and 30% for validation.
• Model performance was evaluated using Brier score, Area Under the Receiver Operating Characteristic (AUC ROC) curve, and calibration plots.
• Predictors included FEV1, BMI, ASA score, and TNM stage in the Epithor model.

Main Results:

• Both models demonstrated similar Brier scores.
• The AUC ROC for the hospital database (0.8) was slightly higher than for Epithor (0.73) on validation data.
• Calibration plots showed a slope less than 1 for both databases, indicating potential over-prediction.

Conclusions:

• Prognostic models derived from national hospital databases show performance comparable to specialized databases.
• National databases may not capture essential clinical variables (e.g., FEV1, ASA, TNM stage), potentially limiting model precision.
• Further research is needed to integrate comprehensive clinical data into national databases for enhanced predictive accuracy.