Reliability of the variables in a new set of models that predict outcome after stroke

To provide valid predictions of outcome, the variables included in a prognostic model must be capable of reliable collection. The authors have recently reported a set of simple but rigorously developed models that predict outcome after stroke. The aim of this study was to establish the inter-rater reliability of the variables included in the models.

Inter-rater agreement was measured prospectively (between two clinicians; 92 patients) and retrospectively (between two auditors; 200 patients) and the validity of the data collected retrospectively was estimated by comparing them with data collected prospectively (195 patients). In the prospective study inter-rater agreement for urinary incontinence and for the variables of three other previously published models was also measured. The median difference (md) between ages and kappa statistics for other variables was calculated.

For the model variables, prospective agreement ranged from good to excellent (age: md 0 years; living alone before the stroke kappa 0.84; pre-stroke functional independence kappa 0.67; normal verbal Glasgow Coma Scale score kappa 0.79; ability to lift both arms against gravity kappa 0.97; ability to walk unaided kappa 0.91) while retrospective agreement (age: md 0 years; kappa 0.55-0.92) and agreement between prospective and retrospective observers (age: md 0 years; kappa 0.49-0.78) was acceptable but less good. Prospective agreement was excellent for urinary incontinence (kappa 0.87) and variable for the other models (kappa 0.23-0.81)

The variables included in these new simple models of outcome after stroke are capable of reliable collection, comparable to or better than that of the other predictive variables considered. When collected retrospectively, the model variables are likely to remain reliable and reasonably valid.