Innovative tools for quality assessment: integrated quality criteria for review of multiple study designs (ICROMS)

With the aim to facilitate a more comprehensive review process in public health including patient safety, we established a tool that we have termed ICROMS (Integrated quality Criteria for the Review Of Multiple Study designs), which unifies, integrates and refines current quality criteria for a large range of study designs including qualitative research.

Review, pilot testing and expert consensus.

The tool is the result of an iterative four phase process over two years: 1) gathering of established criteria for assessing controlled, non-controlled and qualitative study designs; 2) pilot testing of a first version in two systematic reviews on behavioural change in infection prevention and control and in antibiotic prescribing; 3) further refinement and adding of additional study designs in the context of the European Centre for Disease Prevention and Control funded project 'Systematic review and evidence-based guidance on organisation of hospital infection control programmes' (SIGHT); 4) scrutiny by the pan-European expert panel of the SIGHT project, which had the objective of ensuring robustness of the systematic review.

ICROMS includes established quality criteria for randomised studies, controlled before-and-after studies and interrupted time series, and incorporates criteria for non-controlled before-and-after studies, cohort studies and qualitative studies. The tool consists of two parts: 1) a list of quality criteria specific for each study design, as well as criteria applicable across all study designs by using a scoring system; 2) a 'decision matrix', which specifies the robustness of the study by identifying minimum requirements according to the study type and the relevance of the study to the review question. The decision matrix directly determines inclusion or exclusion of a study in the review. ICROMS was applied to a series of systematic reviews to test its feasibility and usefulness in the appraisal of multiple study designs. The tool was applicable across a wide range of study designs and outcome measures.

ICROMS is a comprehensive yet feasible appraisal of a large range of study designs to be included in systematic reviews addressing behaviour change studies in patient safety and public health. The tool is sufficiently flexible to be applied to a variety of other domains in health-related research. Beyond its application to systematic reviews, we envisage that ICROMS can have a positive effect on researchers to be more rigorous in their study design and more diligent in their reporting.