Validation of the extended version of the Implementation Leadership Scale (ILS-X)

Implementation science has made significant advances in our understanding of organizational factors that impact the implementation process. Critical to those advances has been the development of measures of key implementation-focused organizational constructs, such as implementation leadership. The Implementation Leadership Scale (ILS) was developed to capture leadership behavior identified as critical to implementation effectiveness. Recent research in education has identified additional dimensions of implementation leadership that extend our understanding of how leaders contribute to the successful implementation and sustainment of new practices. The goal of this paper is to validate the extended version of the Implementation Leadership Scale, or the ILS-X, in behavioral health settings.

This paper utilized baseline data from two large implementation trials conducted in behavioral health settings that collected survey data on the ILS-X measure from 389 providers across 68 behavioral health clinics. The ILS-X is a pragmatic measure with 21 items assessing seven dimensions of implementation leadership (proactive, knowledgeable, supportive, perseverant, communication, vision/mission, and available). Analyses assessed internal consistency reliability, interrater reliability and agreement, factor structure, and construct validity evidence for scores on the measure.

The ILS-X performed well across all criteria. A second-order confirmatory factor model fit the data well and had high factor loadings across all dimensions. Correlations with other clinic-level measures (e.g., transformational leadership, organizational climate, aggregate job satisfaction, clinic characteristics) were consistent with theory-guided expectations. Internal consistency reliability and aggregation indices supported future use of the measure.

The ILS-X allows implementation researchers and practitioners to reliably assess a more comprehensive array of implementation leadership behaviors that builds on the original ILS measure. The ILS-X will be valuable for targeting an expanded range of behaviors for identifying areas of leadership strength and improvement during implementation efforts.