Methods for stroke severity assessment by chart review in the Atherosclerosis Risk in Communities study

Stroke severity is the most important predictor of post-stroke outcome. Most longitudinal cohort studies do not include direct and validated measures of stroke severity, yet these indicators may provide valuable information about post-stroke outcomes, as well as risk factor associations. In the Atherosclerosis Risk in Communities (ARIC) study, stroke severity data were retrospectively collected, and this paper outlines the procedures used and shares them as a model for assessment of stroke severity in other large epidemiologic studies. Trained physician abstractors, who were blinded to other clinical events, reviewed hospital charts of all definite/probable stroke events occurring in ARIC. In this analysis we included 1,198 ischemic stroke events occurring from ARIC baseline (1987-1989) through December 31, 2009. Stroke severity was categorized according to the National Institutes of Health Stroke Scale (NIHSS) score and classified into 5 levels: NIHSS ≤ 5 (minor), NIHSS 6-10 (mild), NIHSS 11-15 (moderate), NIHSS 16-20 (severe), and NIHSS > 20 (very severe). We assessed interrater reliability in a subgroup of 180 stroke events, reviewed independently by the lead abstraction physician and one of the four secondary physician abstractors. Interrater correlation coefficients for continuous NIHSS score as well as percentage of absolute agreement and Cohen Kappa Statistic for NIHSS categories were presented. Determination of stroke severity by the NIHSS, based on data abstracted from hospital charts, was possible for 97% of all ischemic stroke events. Median (25%-75%) NIHSS score was 5 (2-8). The distribution of NIHSS category was NIHSS ≤ 5 = 58.3%, NIHSS 6-10 = 24.5%, NIHSS 11-15 = 8.9%, NIHSS 16-20 = 4.7%, NIHSS > 20 = 3.6%. Overall agreement in the classification of severity by NIHSS category was present in 145/180 events (80.56%). Cohen's simple Kappa statistic (95% CI) was 0.64 (0.55-0.74) and weighted Kappa was 0.79 (0.72-0.86). Mean (SD) NIHSS score was 5.84 (5.88), with a median score of 4 and range 0-31 for the lead reviewer (rater 1) and mean (SD) 6.16 (6.10), median 4.5 and range 0-36 in the second independent assessment (rater 2). There was a very high correlation between the scores reported in both assessments (Pearson r = 0.90). Based on our findings, we conclude that hospital chart-based retrospective assessment of stroke severity using the NIHSS is feasible and reliable.