Variability in expert assessments of child physical abuse likelihood

In the absence of a gold standard, clinicians and researchers often categorize their opinions of the likelihood of inflicted injury using several ordinal scales. The objective of this protocol was to determine the reliability of expert ratings using several of these scales.

Participants were pediatricians with substantial academic and clinical activity in the evaluation of children with concerns for physical abuse. The facts from several cases that were referred to 1 hospital's child abuse team were abstracted and recorded as in a multidisciplinary team conference. Participants viewed the recording and rated each case using several scales of child abuse likelihood.

Participants (n = 22) showed broad variability for most cases on all scales. Variability was lowest for cases with the highest aggregate concern for abuse. One scale that included examples of cases fitting each category and standard reporting language to summarize results showed a modest (18%-23%) decrease in variability among participants. The interpretation of the categories used by the scales was more consistent. Cases were rarely rated as "definite abuse" when likelihood was estimated at < or = 95%. Only 7 of 156 cases rated < or = 15% likelihood were rated as "no reasonable concern for abuse." Only 9 of 858 cases rated > or = 35% likelihood were rated as "reasonable concern for abuse."

Assessments of child abuse likelihood often show broad variability between experts. Although a rating scale with patient examples and standard reporting language may decrease variability, clinicians and researchers should be cautious when interpreting abuse likelihood assessments from a single expert. These data support the peer-review or multidisciplinary team approach to child abuse assessments.