Evaluation of hybrid stroke quality indicators by integrating NIHSS and claims data for improved outcome prediction

Accurately measuring the quality of stroke care based on claims data alone is challenging. Traditional outcome metrics, e.g. mortality rates, do not capture the effectiveness of critical stroke care processes. We aimed to develop hybrid quality indicators (QIs) by integrating clinical stroke severity data with claims data. Claims data were linked to patient-level clinical data from 15 hospitals (2017-2020) and harmonized in the Observational Medical Outcome Partnership (OMOP) data model. Inclusion criteria, outcomes and risk factors were developed by medical expert panels. We applied machine learning for modeling the outcomes 30-day-mortality, reinfarction within 90 days, and care degree increase within 180 days. We compared extreme gradient boosting (XGBoost) models with and without the National Institutes of Health Stroke Scale (NIHSS) using Receiver-Operating-Characteristic-Area-Under-the-Curve (ROC-AUC) and Brier Score (BS). Hospitals were ranked according to the impact of each QI using Standardized Mortality Ratios (SMRs). The study included 9,348 ischemic (I63) and 1,554 hemorrhagic (I61) strokes, with NIHSS available for 5,012 patients. For all three outcomes, disease severity as measured by NIHSS was the most important determinant. The predictive power of the hybrid models was higher than that of models based on claims data alone. For SMR, the influence of NIHSS was greater than that of age, the most important variable in the claims data model. The results were consistent between the two entities, different outcomes, and sensitivity analyses. Including NIHSS information alongside claims data improves the risk adjustment of quality indicators.