Assessing content validity and user perspectives on the Team Check-up Tool: expert survey and user focus groups

This study assesses content validity and user feedback on the Team Check-up Tool (TCT), an instrument used for measuring dynamic context of quality improvement (QI) teams and their implementation of QI activities.

We conducted two focus groups and one larger feedback session with TCT users to assess feasibility, importance of areas of inquiry and barriers to use. A panel of eight QI experts evaluated the item-by-item content (content validity) of TCT by rating the relevance of each item to implementation success. We calculated item-level and scale-level content validity using the content validity index (CVI).

Scale-level CVI was 0.872. Highly rated items included implementation of recommended interventions, educational activities, team review of performance data, team sharing of performance data with staff and specific barriers to progress. Four items were rated relatively low: presentation of performance data to the hospital/health system board; manner of provision of feedback of data to staff; to what other units the team attempted to spread and turnover of QI team members. Items identified in user focus groups as important included whether there were events distracting staff from the initiative, number of team meetings and turnover of QI team members. Focus groups also identified barriers to the completion of the tool, including lack of feedback and response fatigue during stable activity periods.

The findings support the conclusion that the TCT measures meaningful areas of context and implementation in team-based QI initiatives, particularly intervention activity tracking, review and sharing of performance data and team progress barriers. We offer a modified instrument with a framework for real-time measurement of important elements of implementation and context of QI teams based on the findings.