Development of a Comprehensive Tool to Assess Rigor When Evaluating Quality Improvement Projects

The aim of this study was to develop a pragmatic domain-based tool to Comprehensively Assess Rigor when Evaluating Quality Improvement projects (CARE-QI) that can be used by health professionals, researchers, or academics.

An expert panel was recruited to provide consensus on the tool. The development of the CARE-QI instrument followed a three-stage methodology. Firstly, a scoping review was used to identify potential items for inclusion. Secondly, using these items, a draft version of the tool was developed by the researchers and finally a Delphi survey was initiated to reach consensus on the final items. Two rounds of surveys were required where participants rated their level of agreement with each item on a Likert scale from 1 not important to 5 very important. The final version was sent out in the third round. Participants could provide free text comments on the tool during all rounds.

A total of 40 experts participated in the first round of the Delphi survey. Members consisted of international multi-disciplinary healthcare professionals including clinicians and researchers with an interest in quality improvement and evidence implementation. The final CARE-QI tool consists of 13 items within four-domains: problem and design, context, intervention and implementation, and evaluation and sustainability.

A pragmatic domains-based tool has been developed, in collaboration with experts within the field, to comprehensively assess rigor when evaluating different types of clinical quality improvement projects. Further testing will confirm validity and reliability of the items within the tool.