Related Experiment Videos
Comparative Analysis of Coronary Surgery Risk Stratification Models
Pliam1, Shaw, Zapolanski
1San Francisco Heart Institute, Seton Medical Center, 1900 Sullivan Avenue, Daly City, CA, 94015, USA. 76734.1723@compuserve.com
Insights
This study evaluated coronary bypass surgery (CABG) risk models, finding existing tools predict outcomes with 80% accuracy. However, model predictions varied, suggesting a need for improved comparison methods for CABG surgery.
Area of Science:
- Cardiovascular Surgery
- Health Services Research
- Biostatistics
Background:
- Preoperative risk assessment is crucial for coronary artery bypass grafting (CABG) surgery.
- Existing risk models require validation on independent datasets to ensure generalizability.
- Comparing the performance of various CABG risk stratification tools is essential for clinical utility.
Purpose of the Study:
- To compare the predictive accuracy of established and newly developed risk assessment models for hospital mortality in coronary artery bypass grafting (CABG) patients.
- To evaluate the validity of existing CABG models using a local patient database.
- To assess the variability in mortality predictions across different risk models.
Main Methods:
- Tested four existing models (Parsonnet, Cleveland, STS v1, STS v2) and two new models (Bayesian, Logistic Regression) on a dataset of 3,443 CABG patients.
- Calibrated Parsonnet and Cleveland models using historical data from 2,842 patients operated on before 1991.
- Compared models based on predicted vs. observed mortality, Brier score, C-Index (AUC), and predictive efficiency, including a subset of 3,237 isolated CABG patients.
Main Results:
- Observed mortality was 4.0% for all CABG patients and 3.7% for isolated CABG patients.
- C-Indexes for the models ranged from 0.77 to 0.83, indicating approximately 80% discrimination accuracy.
- Bayesian and Logistic Regression models showed strong performance, with C-Indexes of 0.83 and 0.80 respectively.
- Significant variation in predicted mortality was observed between different models.
Conclusions:
- Existing coronary artery bypass grafting (CABG) risk models demonstrate moderate predictive accuracy (around 80%).
- Models developed using national or non-local databases appear valid for local application.
- Current methods for comparing CABG risk models may be insufficient due to wide prediction variability.
Abstract:
BACKGROUND: Preoperative risk assessment models for coronary bypass surgery (CABG) have been proposed, but comparison of them using independent databases needs to be done. METHODS: Models of CABG hospital mortality were tested on a set of 3,443 patients who underwent CABG including a subset of 3,237 patients who had isolated CABG (no valve procedures), in our database since 1991. Four models previously described were designated as Parsonnet (PS), Cleveland (CL), and Society of Thoracic Surgeons version 1 (ST1) and version 2 (ST2). We developed our own Bayesian (BA) and logistic regression (LR) models and calibrated the PS and CL models on 2,842 patients operated on prior to 1991. Models were compared with respect to 1) mean predicted mortality, 2) correlation of predicted to observed mortality, 3) Brier mean probability score, 4) descriptive statistics, 4) the C-Index (area beneath the receiver operating characteristic curve), and 5) predictive efficiency. Since the ST1 and ST2 models were developed for use only with isolated CABG patients, these models were compared with the others using an isolated CABG subset. RESULTS: Observed mortality for all 3,443 CABG patients was 4.0%. For this group, the mean mortality predicted by PS, CL, BA, LR, was 9.0 +/- 8.0, 6.0 +/- 6.0, 7.6 +/- 15.6, and 5.1 +/- 7.7 (mean +/- standard deviation) respectively. C-Indexes were.80 +/-.02,.80 +/-.02,.83 +/-.02, and.80 +/-.02 (C-Index +/- standard error) respectively. Observed mortality for 3,237 isolated CABG patients was 3.7%. For this subgroup, the mean mortality predicted by PS, CL, BA, LR, ST1, and ST2 was 8.4 +/- 7.4, 5.7 +/- 5.9, 6.5 +/- 13.9, 4.5 +/- 6.5, 9.6 +/- 9.1, and 3.0 +/- 3.3 respectively. C-Indexes were.80 +/-.03,.80 +/-.03,.83 +/-.02,.79 +/-.03,.77 +/-.03, and.81 +/-.02 respectively. CONCLUSIONS: Existing CABG models can accurately discriminate outcome about 80 percent of the time. Models developed on a national database and those from non-local databases appear to have validity for our local data set. Predictions can vary widely between models and existing methods for comparing models appear to be inadequate. The methodology presented here is applicable for use with patients undergoing interventions in the cardiac catheterization laboratory.