Creating and field-testing child maltreatment definitions: improving the reliability of substantiation determinations

The decision to substantiate a case of maltreatment is pivotal to myriad stakeholders; however, the reliability (and therefore, accuracy) of substantiation determinations is suspect. The authors tested if (a) they could develop more reliable substantiation definitions and processes and (b) case workers would be comfortable with and accepting of a new approach. Determinations from five field sites were compared with those of master reviewers (made while listening to the field sites' case presentations). Agreement, sensitivity, specificity, positive predictive values, and negative predictive values were high overall (.87, .96, .97, .89, and .97, respectively) and for each form of maltreatment. Agreement was substantially improved. The definitions and the determination process were well liked by case workers and other stakeholders. Clearly, implementing standardized definitions in a typical child protective environment would pose myriad political challenges. However, our results suggest achieving reliable substantiation determinations may be a feasible goal.