Making a difference: towards a method for weighing the evidence in a qualitative synthesis

In a qualitative synthesis, primary qualitative studies are integrated to develop a theory or evidence-based interventions. Until now, the strength of the evidence in the primary studies has not been taken into account in the outcome of the qualitative synthesis. In this paper, a method is developed and evaluated to assign weights to the findings of the qualitative studies using both the frequency and the quality of the reported results.

Seventeen qualitative studies were retrieved in an illustrative example project on children and trauma. Findings were extracted from the primary studies with the use of coding which resulted in 14 themes. The quality of the studies was appraised using both expert judgement and a quality checklist. These outcomes are used to calculate the weights.

When the outcomes of the checklist appraisal are compared with those of the expert judgement, the effect on the strength of the evidence is virtually always in the same direction. We found that as the frequency with which a concept is studied is low, the strength of the evidence oftentimes decreases even further when using the quality of the results in the weighing process.

In the end, the outcomes of a critical appraisal affect the weight that is placed on particular studies. The use of a checklist is recommended because of its more distinguishing ability. The method that was developed for assigning weights to the evidence is discussed in light of both the quality appraisal in qualitative research and the objectives of qualitative synthesis.