Comparison of Machine Learning Methods With National Cardiovascular Data Registry Models for Prediction of Risk of
Bobak J Mortazavi1,2,3,4, Emily M Bucholz5,6, Nihar R Desai3,4
1Department of Computer Science and Engineering, Texas A&M University, College Station.
Insights
Machine learning models significantly improved prediction of major bleeding after percutaneous coronary intervention (PCI). These advanced techniques help identify high-risk patients for targeted bleeding reduction strategies.
Area of Science:
- Cardiology
- Medical Informatics
- Health Outcomes Research
Background:
- Predicting major bleeding after percutaneous coronary intervention (PCI) is crucial for guiding clinical decisions and minimizing bleeding risk.
- Machine learning (ML) techniques offer potential for enhanced predictive accuracy through sophisticated variable selection.
Purpose of the Study:
- To compare the predictive performance of ML techniques against existing National Cardiovascular Data Registry (NCDR) models for post-PCI major bleeding.
- To evaluate if ML models can more accurately identify patients at risk of bleeding after PCI.
Main Methods:
- A comparative effectiveness study utilizing the NCDR CathPCI Registry data (July 2009–April 2015).
- Machine learning models, including logistic regression with lasso regularization and XGBoost, were developed and validated using 5-fold cross-validation.
- Model performance was compared against recreated NCDR simplified risk score and full models using C statistics, calibration, and F scores.
Main Results:
- The XGBoost model achieved a C statistic of 0.82, outperforming the existing full NCDR model (C statistic of 0.78).
- XGBoost correctly identified an additional 3.7% of high-risk bleeding cases and improved identification of low-risk non-bleeding cases.
- Data-driven decision thresholds enhanced the false discovery rate, improving risk stratification accuracy.
Conclusions:
- Machine learning techniques demonstrate superior prediction of major bleeding after PCI compared to traditional NCDR models.
- These findings suggest ML can significantly aid in identifying patients who would most benefit from bleeding reduction interventions.
Importance:
Better prediction of major bleeding after percutaneous coronary intervention (PCI) may improve clinical decisions aimed to reduce bleeding risk. Machine learning techniques, bolstered by better selection of variables, hold promise for enhancing prediction.
Objective:
To determine whether machine learning techniques better predict post-PCI major bleeding compared with the existing National Cardiovascular Data Registry (NCDR) models.
Design, Setting, And Participants:
This comparative effectiveness study used the NCDR CathPCI Registry data version 4.4 (July 1, 2009, to April 1, 2015), machine learning techniques were used (logistic regression with lasso regularization and gradient descent boosting [XGBoost, version 0.71.2]), and output was then compared with the existing simplified risk score and full NCDR models. The existing models were recreated, and then performance was evaluated through additional techniques and variables in a 5-fold cross-validation in analysis conducted from October 1, 2015, to October 27, 2017. The setting was retrospective modeling of a nationwide clinical registry of PCI. Participants were all patients undergoing PCI. Percutaneous coronary intervention procedures were excluded if they were not the index PCI of admission, if the hospital site had missing outcomes measures, or if the patient underwent subsequent coronary artery bypass grafting.
Exposures:
Clinical variables available at admission and diagnostic coronary angiography data were used to determine the severity and complexity of presentation.
Main Outcomes And Measures:
The main outcome was in-hospital major bleeding within 72 hours after PCI. Results were evaluated by comparing C statistics, calibration, and decision threshold-based metrics, including the F score (harmonic mean of positive predictive value and sensitivity) and the false discovery rate.
Results:
The post-PCI major bleeding rate among 3 316 465 procedures (patients' median age, 65 years; interquartile range, 56-73 years; 68.1% male) was 4.5%. The existing full model achieved a mean C statistic of 0.78 (95% CI, 0.78-0.78). The use of XGBoost and full range of selected variables achieved a C statistic of 0.82 (95% CI, 0.82-0.82), with an F score of 0.31 (95% CI, 0.30-0.31). XGBoost correctly identified an additional 3.7% of cases identified as high risk who experienced a bleeding event and an overall improvement of 1.0% of cases identified as low risk who did not experience a bleeding event. The data-driven decision threshold helped improve the false discovery rate of the existing techniques. The existing simplified risk score model improved the false discovery rate from more than 90% to 78.7%. Modifying the model and the data decision threshold improved this rate from 78.7% to 73.4%.
Conclusions And Relevance:
Machine learning techniques improved the prediction of major bleeding after PCI. These techniques may help to better identify patients who would benefit most from strategies to reduce bleeding risk.
More Related Videos
06:19Constructing and Visualizing Models using Mime-based Machine-learning Framework
Published on: July 22, 2025
04:09Predicting Treatment Response to Image-Guided Therapies Using Machine Learning: An Example for Trans-Arterial Treatment of Hepatocellular Carcinoma
Published on: October 10, 2018
Related Concept Videos
01:23Assessment of the Cardiovascular System I: Subjective Data
Initial Enquiry
Ask the patient about their primary concern and thoroughly explore all reported symptoms.
Medical History
Investigate past illnesses affecting the cardiovascular system, such as angina, anemia, rheumatic fever, congenital heart disease, stroke, thrombophlebitis, dysrhythmias, varicosities
Inquire about symptoms...
02:57The Sense of Self: Reflected Self-Appraisal and Social Comparison
01:26National Nursing Organizations I
01:14Analysis Methods of Pharmacokinetic Data: Model and Model-Independent Approaches
The model approach uses mathematical models to describe changes in drug concentration over time. Pharmacokinetic models help characterize drug behavior in patients, predict drug concentration in the body fluids, calculate optimum dosage regimens, and evaluate the risk of toxicity. However, ensuring that the model fits the experimental data accurately...
01:30National Nursing Organizations II
The AACN emphasizes a healthy work environment through six standards to achieve an optimal patient outcome. The standards are appropriate staffing, meaningful recognition, collaboration, authentic leadership, effective communication, and decision-making. In addition, AACN provides certification programs, webinars, journals, and...
01:30Simplified Synchronous Machine Model
In this model, each generator is connected to a...