Framingham Risk Score Prediction at 12 Months in the STANDFIRM Randomized Control Trial

The STANDFIRM (Shared Team Approach Between Nurses and Doctors for Improved Risk Factor Management; ANZCTR registration ACTRN12608000166370) trial was designed to test the effectiveness of chronic disease care management for modifying the Framingham risk score (FRS) among patients with stroke or transient ischemic attack. The primary outcome of change in FRS was not met. We determine baseline characteristics that predict reduction in FRS at 12 months and whether future FRS is predetermined at baseline.

We used machine learning regression methods to evaluate 35 variables encompassing demographics, risk factors, psychological, social and education status, and laboratory tests. We determine the optimal machine learning and associated tuning parameters from the following: random forest, extreme gradient boosting, category boosting, support vector regression, multilayer perceptron neural network, and K-nearest neighbor. Training (n=404) and test (n=103) data were evenly matched for age, sex, baseline, and 12-month FRS. The optimal model for predicting FRS at 12 months was category boosting (R²=0.712; root mean square error, 7.32). The 5 variables with highest Shapley values for category boosting were baseline FRS (Shapley additive explanation [SHAP], 8.42 of total of 12.12), age (SHAP, 1.58), systolic blood pressure (SHAP, 0.23), male sex (SHAP, 1.05), and London Handicap (SHAP, 0.18). Machine learning methods were poor at determining change in FRS at 12 months (R²<0.22).

Our findings suggest that change in FRS as an end point in secondary stroke trials may have limited value as it is largely determined at baseline. In this cohort, category boosting was the optimal method to predict future FRS but not change in FRS.