Reliability and validity of child passenger safety restraint observations by community observers

To compare the accuracy of trained community observers for direct observation of child passenger restraint use to certified child passenger safety technicians who are either professional observers or community-based technicians, and to compare these three groups with a gold standard.

This is a cross-sectional study of interobserver agreement and accuracy in which 75 photos of children depicted in different child passenger restraint systems were rated by 9 observers total, with 3 representing professional observers, 3 representing certified child passenger safety technicians, and 3 representing trained community observers. For each photo, observers indicated type of restraint; the appropriateness of the harness, if applicable; and overall appropriateness of the restraint. A gold standard was established by consensus agreement of 2 certified car seat technician instructors.

The sensitivity and specificity for trained community observers in identifying broad groupings of restraint types was good (78-100% sensitivity; 93-99% specificity), but they had low agreement with the gold standard for overall appropriateness of the child passenger restraints (kappa = 0.28). The community observer group was 42 percent less likely to code the photo depiction of appropriate restraint use as appropriate.

Community trained observers do show good sensitivity and specificity for identifying the type of restraint but have a trend toward poorer judgment when determining harness appropriateness and overall appropriateness. They may be a cost-effective option for limited restraint identification.