Development of a survey to support assessment of safety learning systems

Patient safety learning systems play a critical role in supporting safety culture in healthcare organisations. A lack of explicit standards leads to inconsistent implementation across organisations, causing uncertainty about their roles and impact. Organisations can address inconsistent implementation by using a self-assessment tool based on agreed-on best practices. Therefore, we aimed to create a survey instrument to assess an organisation's approach to learning from safety events.

The foundation for this work was a recent systematic review that defined features associated with the performance of a safety learning system. We organised features into themes and rephrased them into questions (items). Face validity was checked, which included independent pre-testing to ensure comprehensibility and parsimony. It also included clinical sensibility testing in which a representative sample of leaders in quality at a large teaching hospital (The Ottawa Hospital) answered two questions to judge each item for clarity and necessity. If more than 20% of respondents judged a question unclear or unnecessary, we modified or removed that question accordingly. Finally, we checked the internal consistency of the questionnaire using Cronbach's alpha.

We initially developed a 47-item questionnaire based on a prior systematic review. Pre-testing resulted in the modification of 15 of the questions, 2 were removed and 2 questions were added to ensure comprehensiveness and relevance. Face validity was assessed through yes/no responses, with over 80% of respondents confirming the clarity and 85% the necessity of each question, leading to the retention of all 47 questions. Data collected from the five-point responses (strongly disagree to strongly agree) for each question were used to assess the questionnaire's internal consistency. The Cronbach's alpha was 0.94, indicating a high internal consistency.

This self-assessment questionnaire is evidence-based and on preliminary testing is deemed valid, comprehensible and reliable. Future work should assess the range of survey responses in a large sample of respondents from different hospitals.