Scoping review of instruments measuring organizational context in implementation studies within healthcare

Healthcare providers' use of research in practice is suboptimal. In Learning Health Systems, using the best available research evidence to guide practice is fundamental to continuous improvement. Whilst there is a growing body of evidence that context modifies the effects of implementation strategies to promote research use, knowledge on how best to measure context is lacking. As part of a larger research programme, which involved development of the Implementation in Context (ICON) framework comprising six domains and 22 attributes (Fig. 1), we conducted a scoping review to identify psychometrically sound and pragmatic instruments for measuring macro- or meso-level ICON attributes.

The review was conducted in two phases: (1) identification of instruments that align with the 20 meso- and macro-level attributes in ICON and (2) assessment of their psychometric and pragmatic properties. Instruments and psychometric papers were identified through five electronic databases. The COnsensus-based Standards for selecting health Measurement INstruments (COSMIN) criteria guided the psychometric evaluation. A modified GRADE approach was used to summarize the quality of the evidence for each instrument. In this paper, we report the properties of the included instruments measuring one of the macro- or meso-level attributes.

We identified 38 instruments (and assessed 169 psychometric papers) that captured nine of 15 meso-level ICON attributes; instruments measuring macro-level ICON attributes were not located. Most of the 38 instruments were rated as inadequate or doubtful (n = 31) on content validity. Using the COSMIN recommendation ratings, 13 instruments are recommended for measuring five meso-level ICON attributes: organizational climate, organizational change processes, receptivity to change, evaluation activity, and leadership. The remaining 25 instruments have the potential for use. All 38 instruments were rated to be at least adequate on pragmatic criteria.

We identified 13 instruments that measure meso-level ICON attributes that influence research implementation and meet psychometric quality criteria. Future work will include: (1) additional psychometric assessments of the 25 instruments not yet recommended for use, (2) psychometric and pragmatic assessments of instruments that assess multiple attributes in ICON, and (3) the development of instruments to measure ICON attributes for which instruments were not identified. Registration Open Science Framework, 21 June 2022, https://doi.org/10.17605/OSF.IO/B2Z8K.