A systematic approach to conducting a non-statistical meta-analysis of research literature

Literature analyses and syntheses are becoming increasingly important as a means of periodically bringing coherence to a research area, contributing new knowledge revealed by integrating single studies, and quickly informing scientists of the state of the field. As a result, there is a need for approaches that can provide replicable, reliable, and trustworthy results. Over the last decade many researchers have begun using the statistical meta-analysis approach to integrate studies. However, the single studies conducted in many areas are not of the type amenable to statistical meta-analysis but are more appropriate for non-statistical analysis and synthesis. The present paper describes (1) a rigorous approach to conducting a non-statistical meta-analysis of research literature and (2) an example of how this approach was applied to the literature of determinants of primary care specialty choice published between 1987 and 1993. This approach includes model development, literature retrieval, literature coding, rating references for quality, annotating high-quality references, and synthesizing only the subset of the literature found of sufficient quality to be considered. Also, the basic results of each included study are reported in the synthesis so that readers have before them all the "data points" used in the synthesis. Thus, readers can draw their own interpretations without having to re-collect the data, just as they would be able to do in any single study that presents original data as well as conclusions and discussion.