Development of an early intervention in psychosis services fidelity questionnaire

We describe the development of an online fidelity questionnaire for early intervention in psychosis (EIP) services, to be used in population-level research, and that can be completed using self-reports from EIP staff.

A review of key literature sources on the components of EIP services was used to identify those components eligible for inclusion in the questionnaire. A modified Delphi approach, using experts in EIP services, was used to select the most important components to include in the questionnaire. To pilot test the questionnaire, two EIP staff members completed one fidelity questionnaire each, and a third questionnaire was completed by an external rater. Responses from the three sources were compared and used to revise the fidelity questionnaire.

Twenty-two experts from England and Canada responded to two Delphi rounds, identifying the top 25 most important EIP service components. Some evidence-based components were not rated as highly as some non-evidence-based components. Pilot testing showed that the EIP staff rated fidelity higher than the external rater. Several questions were removed and/or revised based on the pilot study findings.

Fidelity instruments are limited by the available evidence and the personal experiences of experts used to develop them. As such, fidelity instruments and EIP services should continually be updated to reflect new knowledge. The online fidelity questionnaire was a simple and efficient way to collect data. Future evaluations of the fidelity questionnaire need to ensure that externally collected fidelity data are comprehensive and accurate.