Evaluating intensive family preservation programs: a methodological review

To determine the adequacy of evaluations of family preservation services (FPS), which are designed to support families and prevent out-of-home placements of children at risk of abuse or neglect, and to assess the effectiveness of FPS at reducing out-of-home placements of children.

References published from 1977 to 1993 were identified from a computerized search of databases for English-language publications using the key phrases "family preservation," "child abuse," and "family-based services." Unpublished references were identified by mail or phone from a listing of more than 200 programs in a national directory.

Of 802 references initially identified, 46 program evaluations were reviewed. Ten studies met the following inclusion criteria: (1) evaluated an intensive family preservation program, (2) included outcome data in the report, and (3) used a comparison group. Five were randomized trials, and 5 were quasi-experimental studies (nonrandomized).

Descriptive information about the programs and evaluations was collected. To determine methodological quality, two independent raters used a 15-item questionnaire to examine the assignment of families to treatment groups, the interventions provided, and the outcomes assessed. A composite score of 11 or greater represented an acceptable study, 6 to 10 represented an adequate study, and 5 or less represented an unacceptable study.

Only two studies were rated acceptable, four were adequate, and four were unacceptable. Methodological shortcomings included poorly defined assessment of risk, inadequate descriptions of the interventions provided, and nonblinded determination of the outcomes. Rates of out-of-home placements were 21% to 59% among families who received FPS and 20% to 50% among comparison families. The relative risk of placement was significantly reduced by FPS in only two studies (one randomized trial and one quasi-experimental study).

Despite current widespread use of FPS to prevent out-of-home placements of children, evaluations of FPS are methodologically difficult and show no benefit in reducing rates of out-of-home placements of children at risk of abuse or neglect in 8 of 10 studies. Consistent, methodologically rigorous evaluations are needed to determine the effectiveness of FPS and to guide social policy for high-risk children and their families.