Initial testing of an electronic application of the simplified modified Rankin Scale questionnaire (e-smRSq)

To facilitate modified Rankin scale (mRS) assessments, we developed and tested a smartphone/web application of the simplified mRS questionnaire (e-smRSq). The e-smRSq guides raters towards a final score according to the smRSq algorithm, and offers hints for scoring based on the conventional mRS concepts.

Initially, three experienced mRS certified raters prepared 30 vignettes of unstructured stroke patient interviews, and determined consensus reference scores. Using the e-smRSq, 16 raters of varied professional backgrounds without mRS training scored the mRS for 24 randomly selected vignettes. Subsequently, 5 certified and 5 uncertified raters using the e-smRSq scored 23 mRS certification vignettes developed and used in the Strategies to Innovate Emergency Care Clinical Trials Network-Neurological Emergencies Treatment Trials (SIREN-NETT). Cohen's and Fleiss's kappa (κ), weighted kappa (κw), and intra-class correlation (ICC) compared rater scores with reference scores and assessed interrater reliability.

For the 16 initial raters using the e-smRSq with 24 vignettes, the κ (Fleiss) was 0.62 and ICC 0.87 (CI 0.80-0.93). Comparing raters' scores with reference scores, Cohen's κ was 0.68 and κw 0.90. For the 10 subsequent raters using the e-smRSq on SIREN-NETT vignettes, κ (Fleiss) was 0.8 and ICC 0.95 (CI 0.91-0.97). Comparing all 10 raters scores with SIREN-NETT reference scores, Cohen's κ was 0.88 and κw 0.97. There was no significant difference between certified and uncertified raters.

The e-smRSq appears to have good reproducibility and validity metrics among both certified and non-certified mRS raters, possibly owing to its simplicity. Further testing in stroke patients in warranted.