Related Experiment Video
Updated: Jun 20, 2026
The Mouse Stroke Unit Protocol with Standardized Neurological Scoring for Translational Mouse Stroke Studies
Published on: February 7, 2025
A scale for distinguishing efficacy from effectiveness was adapted and applied to stroke rehabilitation studies
Laura L Zettler1, Mark R Speechley, Norine C Foley
1Department of Epidemiology and Biostatistics, Schulich School of Medicine and Dentistry, The University of Western Ontario, London, Ontario, Canada. lauraz@chatham-kent.ca
Objectives:
To apply a tool that purports to differentiate between efficacy and effectiveness studies to stroke rehabilitation trials and to evaluate its applicability and reliability.
Study Design And Setting:
Three raters developed item operational definitions before independently applying the seven-item scale to 151 randomized controlled trials (RCT), published during or after 1997, that evaluated either a pharmacologic (P, n=78) or a nonpharmacologic (NP, n=73) intervention. Inter-rater reliability was assessed for both individual items and total scores, separately for P and NP trials.
Results:
Item inter-rater reliability (multiple-rater kappa) ranged from 0.00 (95% CI [confidence interval]: -0.13, 0.13) to 0.85 (95% CI: 0.73, 0.98) and from 0.21 (95% CI: 0.08, 0.34) to 0.79 (95% CI: 0.66, 0.92) for P and NP RCTs, respectively. For the total score (dichotomized), kappa values were 0.43 (95% CI: 0.31, 0.56) and 0.51 (95% CI: 0.37, 0.64) for P and NP trials, respectively.
Conclusions:
The tool provides a solid foundation on which to base further discussion of the differential criteria of efficacy-effectiveness trial design. Scale items should be properly operationalized depending on the research question of interest and evaluated for reliability before the scale is used for definitively judging a given study's design or the external validity of its results.
