A critical appraisal of meta-analysis

Meta-analysis is an increasingly popular, objective method for summarizing a body of empirical findings. The standard meta-analysis package consists of methods for estimating the combined probability and average effect size for a set of studies, the stability of these results, and the factors associated with differential treatment outcomes. While meta-analysis is a powerful analytic technique, the procedure has limitations that should be carefully evaluated when it is applied to the psychotherapy--or any other--literature. These limitations include biased selection of studies; reporting inaccuracies, poor quality data, various sources of invalidity (conceptual, methodological, and statistical), and lack of independence in the studies reviewed; and variability in outcome produced by the meta-analytic techniques employed. Despite these potential problems, the advantages of meta-analysis are so substantial that the techniques deserve routine use as an aid to summarizing treatment literatures.