Aggregation of qualitative studies--From theory to practice: Patient priorities and family medicine/general practice...

Aggregation (i.e., meta-ethnography or meta-synthesis) of qualitative studies remains relatively rare and controversial. We have attempted this procedure within an investigation of patient priorities and evaluations of primary care in order to triangulate an instrument development process as well as explore associated dilemmas.

The procedures included a literature search of qualitative research on patient priorities and evaluations and creation of a framework for quality assessment of retrieved papers. The tool for the evaluation of quality in qualitative studies was piloted, refined, and applied to the retrieved literature. The articles were equally distributed between two teams in random fashion, and inter-rater agreement calculated. Finally, we formulated and applied a strategy for aggregation of data from included papers that allowed comparison to a systematic review of quantitative studies on the topic.

Thirty-seven articles met inclusion criteria. Twenty-four of these articles were of sufficient quality to be included in the qualitative aggregation. Inter-rater agreement ranged from 0.22 to 0.77 and 0.38 to 0.60 for pair and assessor comparisons, respectively. The aggregation strategy enabled synthesis within sub-categories of the heterogeneous papers.

We have devised a modestly reliable instrument to assess the quality of qualitative work. The procedure for quality assessment and aggregation appears to be both feasible and potentially useful, though both theoretical and practical problems underline the need for further refinement prior to widespread utilization of this approach.

An instrument to assess the quality of qualitative work within the context of aggregation efforts is described. Calculating inter-rater reliability in this framework can support future quality assessments. A method of breaking a heterogeneous collection of included papers into sub-categories to enable aggregation of qualitative studies is applied and demonstrates its feasibility and potential usefulness.