The Vancouver sedative recovery scale for children: validation and reliability of scoring based on videotaped...

We originally developed and tested the Vancouver Sedative Recovery Scale (VSRS) to measure recovery from sedation following paediatric open heart surgery and reported excellent clinical inter-observer reliability. We now report a new study using videotape instruction of novice raters and videotaped case examples to determine whether the instruction produces adequate skill with the VSRS. Inter-rater reliability was assessed using videotapes of 16 children across a range of ages (six months to six years), and all levels of sedation (unresponsive to fully awake). Variably randomized subsets of six of the 16 test cases were randomly assigned to be rated by each of 16 video-instructed ICU staff volunteers, according to a balanced incomplete block design, such that every pair of raters assessed two children in common. The validity of the ratings from the video-instructed raters was assessed by comparison with "gold standard" scores from two experts who rated all 16 children clinically as the test cases were videotaped. The experts were in agreement themselves (intraclass correlation of 0.976). The correlation between the novice scores (average of six ratings per video) and the live clinical scores (average of two expert ratings) was 0.977 over the 16 test cases. On average, the mean expert rating was slightly higher, but the difference was negligible. (The differences between the mean ratings of the experts and novices for 13 of the 16 videos are very close to zero, while the other three differences, from technically less good videos, are two standard deviations away from zero). The VSRS, video instruction method and incomplete block design may be of use to other clinicians and investigators.