An automated flowchart for the Modified Rankin Scale assessment: A multicenter inter-rater agreement analysis

The Modified Rankin Scale (mRS) is a widely adopted scale for assessing stroke recovery. Despite limitations, the mRS has been adopted as primary outcome in most recent clinical acute stroke trials. Designed to be used by multidisciplinary clinical staff, the congruency of this scale is not consistent, which may lead to mistakes in clinical or research application. We aimed to develop and validate an interactive and automated digital tool for assessing the mRS-the iRankin.

A panel of five board-certified and mRS-trained vascular neurologists developed an automated flowchart based on current mRS literature. Two international experts were consulted on content and provided feedback on the prototype platform. The platform contained five vignettes and five real video cases, representing mRS grades 0-5. For validation, we invited neurological staff from six comprehensive stroke centers to complete an online assessment. Participants were randomized into two equal groups usual practice versus iRankin. The participants were randomly allocated in pairs for the congruency analysis. Weighted kappa (kw) and proportions were used to describe agreement.

A total of 59 professionals completed the assessment. The kw was dramatically improved among nurses, 0.76 (95% confidence interval (CI) = 0.55-0.97) × 0.30 (0.07-0.67), and among vascular neurologists, 0.87 (0.72-1) × 0.82 (0.66-0.98). In the accuracy analysis, after the standard mRS values for the vignettes and videos were determined by a panel of experts, and considering each correct answer as equivalent to 1 point on a scale of 0-15, it revealed a higher mean of 10.6 (±2.2) in the iRankin group and 8.2 (±2.3) points in the control group (p = 0.02). In an adjusted analysis, the iRankin adoption was independently associated with the score of congruencies between reported and standard scores (beta coefficient = 2.22, 95% CI = 0.64-3.81, p = 0.007).

The iRankin adoption led to a substantial or near-perfect agreement in all analyzed professional categories. More trials are needed to generalize our findings. Our user-friendly and free platform is available at https://www.irankinscale.com/.