Related Experiment Videos
External Validation of the BEST-CLI Major Adverse Cardiac Risk Calculator in the Vascular Quality Initiative and
Nathan T P Patel1, Gregory Mouradian1, Mead B Ferris1
1Division of Vascular Surgery and Endovascular Therapy, University of Vermont Medical Center, Burlington, VT.
Insights
The BEST-CLI major adverse cardiac event (MACE) model shows modest predictive ability in real-world vascular practice. External validation revealed the model underestimated one-year MACE by 8.6% on average.
Area of Science:
- Cardiovascular Medicine
- Health Services Research
- Medical Informatics
Background:
- The BEST-CLI cohort developed a predictive model for one-year major adverse cardiac events (MACE).
- External validation is crucial to assess model generalizability in diverse populations and settings.
- Real-world data from the vascular quality initiative (VQI) offers a valuable resource for such validation.
Purpose of the Study:
- To externally validate the BEST-CLI one-year MACE prediction model.
- To assess the model's performance on a real-world database (VQI) using Medicare-linked outcomes.
- To evaluate the model's calibration and discrimination in an independent cohort.
Main Methods:
- Utilized the vascular quality initiative (VQI) database with Medicare-linked long-term outcomes.
- Compared observed one-year MACE in VQI patients against predictions from the BEST-CLI model.
- Employed calibration modeling and C-statistics to evaluate model performance.
Main Results:
- The mean deviation between observed and predicted one-year MACE in the VQI cohort was 8.6% (95% CI 8.5% - 8.7%), indicating underestimation.
- Model calibration revealed a slope of 0.54 and an intercept of 0.31.
- Model discrimination, assessed by the C-index, was 0.62.
Conclusions:
- The BEST-CLI one-year MACE model demonstrates modest predictive ability in real-world vascular practice.
- The model tends to underestimate the occurrence of one-year MACE by approximately 8.6% on average.
- Further refinement or recalibration may be necessary for optimal performance in diverse real-world settings.
Background:
To perform external validation of the 1-year major adverse cardiac event (MACE) model derived from the Best Endovascular versus Best Surgical Therapy in patients with critical limb ischemia (BEST-CLI) cohort on the real-world database from the vascular quality initiative (VQI) and Vascular Implant Surveillance and Interventional Outcomes Network (VISION).
Methods:
We applied the BEST-CLI MACE model to compare the observed versus expected rate of 1-year MACE defined as death, myocardial infarction (MI), and cerebrovascular accident (CVA) in patients within the VQI-VISION Peripheral Vascular Intervention and Infrainguinal Registries (2016-2019). Kaplan-Meier analysis was performed to calculate the observed 1-year rate of MACE. Model calibration and discrimination were assessed with Cox calibration slope and intercept with Harrel's C-index.
Results:
We identified 19,144 eligible patients (endovascular n=14,878, 77.7%; open n=4,266, 22.3%) from the VQI-VISION database. Kaplan-Meier estimated 1-year mortality was 25.3% (95% confidence interval [CI] 24.6%-25.9%), MI 10.3% (95% CI 9.8%-10.8%), and CVA 4.9% (95% CI 4.6%-5.3%), with a composite MACE of 32.6% (95% CI 31.9%-33.3%). BEST-CLI MACE scores ranged between -4 and 23 (median patient score = 9, interquartile range: 5-13). The mean deviation between the observed 1-year MACE from the VQI-VISION compared to BEST-CLI estimated MACE was 8.6% (95% CI 8.5%-8.7%) higher. The model calibration slope was 0.54, intercept was 0.31, and discrimination C-index was 0.62.
Conclusion:
This external validation of the BEST-CLI 1-year MACE scoring system suggests modest predictive ability in real-world vascular practice with the model underestimating 1-year MACE on average by 8.6%.
Related Concept Videos
01:25Imaging Studies for Cardiovascular System VI: Calcium -Scoring CT
01:30Acute Coronary Syndrome III: Diagnostic Studies
01:21Peripheral Arterial Disease II: Clinical Manifestations and Diagnostic Evaluation
01:18Cardiovascular Drugs: Classification based on Therapeutic Indications
01:23Assessment of the Cardiovascular System I: Subjective Data
Initial Enquiry
Ask the patient about their primary concern and thoroughly explore all reported symptoms.
Medical History
Investigate past illnesses affecting the cardiovascular system, such as angina, anemia, rheumatic fever, congenital heart disease, stroke, thrombophlebitis, dysrhythmias, varicosities
Inquire about symptoms...
01:21Imaging Studies for Cardiovascular System IV: CMRI