Machine learning redevelopment of GRACE, ACEF, and TIMI scores for 6-month mortality

In recent years, advancements in our understanding of the pathophysiological mechanisms underlying coronary artery disease (CAD) have introduced new challenges regarding the clinical application of traditional risk scores. While studies suggest that machine learning (ML) algorithms surpass traditional statistical methods in risk prediction, their conclusions are often derived from heterogeneous datasets and varying model structures, which restrict their generalizability and persuasive power.

This study aims to evaluate the clinical performance of the GRACE, ACEF, and TIMI risk scores, while also enhancing their predictive accuracy through redevelopment utilizing six ML algorithms.

This retrospective study was undertaken at the First Hospital of Lanzhou University between January 1, 2019, and December 31, 2020. Six ML algorithms were employed to redevelop the original GRACE, TIMI, and ACEF risk scores. Model performance was evaluated using accuracy, sensitivity, precision, F1-score, the area under the receiver operating characteristic curve (AUROC), and the area under the precision-recall curve (AUPRC).

We retrospectively enrolled 1,682 patients diagnosed with acute myocardial infarction (AMI) who underwent emergency percutaneous coronary intervention (PCI). The derivation cohort from 2019 included 883 patients, among whom 48 (5.4%) experienced death during the follow-up period. The temporal validation cohort from 2020 consisted of 799 patients, with 26 (3.25%) reported deaths. The original GRACE, TIMI, and ACEF scores were validated using derivation data, resulting in AUROC values of 0.84, 0.68, and 0.69, and AUPRC values of 0.49, 0.19, and 0.34, respectively. The redevelopment of ML algorithms significantly enhanced predictive performance, with the ensemble algorithms yielding the most favorable results. The redeveloped GRACE, TIMI, and ACEF models achieved AUROC values of 0.89, 0.75, and 0.91, and AUPRC values of 0.55, 0.50, and 0.50, respectively. Demonstrating superior predictive performance for six-month mortality over the redeveloped ACEF and TIMI scores, the redeveloped GRACE score yielded an AUROC of 0.89 and an AUPRC of 0.32 in temporal validation.

This study validated three established risk scores and found that the GRACE score exhibited superior predictive performance for six-month mortality compared to the ACEF and TIMI scores. The redevelopment of these scores using ML techniques significantly enhanced their predictive accuracy. This improvement reflects both the inherent strengths of ML algorithms and the recalibration of parameter weights to better align with contemporary clinical practice.