The uses and abuses of meta-analysis

Meta-analysis is a quantitative process of summary and interpretation which involves pooling information from independent studies concerning a single theme in order to draw conclusions. Greatly increased employment of meta-analysis is currently being advocated for clinical and policy decision making. However, the prestige of meta-analysis is based upon a false model of scientific practice. Interpreting empirical research is an extremely complex activity requiring clinical and scientific knowledge of the field in question; and teams of professional 'meta-analysts' with a primary skill base in information technology and biostatistics cannot take over this role. Meta-analysis is not a hypothesis-testing activity, and cannot legitimately be used to establish the reality of a putative hazard or therapy. The proper use of meta-analysis is to increase the precision of quantitative estimates of health states in populations. If used to estimate an effect, the reality of that effect should have been established by previous scientific studies. But the summary estimate from a meta-analysis can only be directly applied to a target population when the 'meta-protocol' and 'meta-population' match the target situation in all relevant particulars. These constraints can rarely be satisfied in practice, so the results of meta-analysis typically require adjustment--which is a complex, assumption-laden process that negates many of the statistical power advantages of a meta-analysis. Lacking any understanding or acknowledgement of the need for adjustment, most meta-analyses must be regarded as abuses of the technique.