Assessing child maltreatment prevention via administrative data systems: A case example of reproducibility

Critical issues about scientific reproducibility have been raised about biomedical research, including the reliability of data and analyses within a given study. The case example in this article examined a reproducibility issue pertaining to the use of administrative data systems for evaluation of child maltreatment (CM) prevention, making use of a prevention study conducted over a decade ago that provided a unique opportunity. The place-randomization study, which randomized counties to condition, found that community-wide implementation of a parenting and family support intervention produced positive impact on county-wide rates for substantiated CM cases and out-of-home placements, documented through a state information system. The key consideration is whether and to what extent the administrative record data re-examined retroactively a decade later for the original study's time period would yield comparable results to those based on data acquired at the time of the study. The results indicated that despite small changes over time, the same data patterns and statistical effects were reproducible for the two archival outcome variables. For substantiated CM, the reproduced analyses reflected higher effect sizes and a clear pattern of reduction as a function of intervention. For out-of-home placements, effect sizes were quite comparable to the original ones, reflecting preventive impact. Overall, this case study illustrated the verifiability of data reproducibility in the context of a population outcome evaluation, which underscores the importance of reliable population-prevalence measurement as an essential part of a comprehensive public health strategy aimed at the prevention of CM.