Construct validity, reliability and measurement invariance of the intervention usability scale - insights from two...

Usability has been proposed as a critical factor influencing the implementation of psychological interventions in routine practice. The Intervention Usability Scale (IUS) has previously been adapted from the System Usability Scale to assess the perceived usability of psychological interventions. However, the construct validity, reliability and theoretical consistency of the IUS remain unclear. This study evaluates the psychometric properties of the Finnish version of the IUS in two cognitive-behavioral therapy-based interventions delivered in primary healthcare.

A translated Finnish version of the IUS was administered to healthcare professionals trained in either low-intensity guided self-help (n = 921) or high-intensity face-to-face cognitive-behavioral therapy programs (n = 455). Exploratory and confirmatory factor analyses were conducted to examine the underlying structure of the IUS. Configural, metric and scalar invariance was assessed across multiple demographic subgroups. Reliability was assessed using McDonald's Omega.

Exploratory analyses suggested a three-factor structure, contradicting previous findings supporting a unidimensional or two-factor models. Confirmatory factor analysis in the guided self-help dataset showed moderate model fit after treating identified issues with correlated residuals and item cross-loadings. Measurement invariance was established across all assessed demographic characteristics. However, the identified factor structure did not replicate in the face-to-face cognitive-behavioral therapy dataset, raising concerns about the generalizability of the measure across different interventions. Reliability estimates were acceptable in the guided self-help dataset but weaker in the face-to-face cognitive-behavioral therapy dataset.

The findings suggest that the IUS does not robustly measure usability as previously proposed. The measure's factor structure varies across different interventions, and its alignment with theoretical models of usability is unclear. These results question the applicability of the IUS for assessing usability in psychological interventions. If usability is to be measured in this context, a domain-specific approach may be necessary. A roadmap for further measure development is outlined. Until further validation is conducted, the use of the IUS for assessing psychological interventions to guide intervention implementation is not recommended.