Development and validation of the Dartmouth Operative Conditions Scale

Studies of pediatric sedation practice have suffered from the lack of an objective scale that would allow for a comparison of the effectiveness and safety of sedation provided by various providers and techniques. We present the Dartmouth Operative Conditions Scale (DOCS), which is designed as a research tool to codify the appropriateness of the procedural conditions provided by various sedation interventions. To begin, human factors methodology was used to develop a model of the pediatric sedation process and to define the criteria for measuring a patient's condition during a procedure (DOCS). To accomplish validation, 70 video clips (30-s duration) were then selected from more than 300 h of procedural video tape for testing/grading purposes. Inter-rater reliability was tested by comparing the score for each video clip among 10 different raters. Intra-rater reliability was evaluated by retesting all of the raters 1 yr after their initial rating. Construct validity was confirmed by analyzing the change in DOCS score relative to the time that sedation intervention was undertaken. Criterion validity was tested by comparing the DOCS to a modified COMFORT score. The DOCS was completed with excellent inter-rater (kappa = 0.84) and intra-rater (kappa = 0.91) agreement by 10 health care providers with various backgrounds during the 1-yr study period. Criterion validity was supported by the close correlation between the DOCS and the modified COMFORT scores for 20 distinct video clips (Spearman correlation coefficient = 0.98; P <0.001). The distribution of DOCS scores 20 min after the anesthetic induction was significantly lower than the scores before initiation of sedation, and scores after emergence were consistently higher than those 20 min after sedation (P <0.001), thus confirming construct validity of the scale. The DOCS is a validated research tool when used with video data for comparing the effectiveness and safety of pediatric sedation service, regardless of technique used for decreasing anxiety or pain during a procedure.