Getting SSPOT to run: Development of a novel direct observational tool for usage in clinical settings

Limited literature exists describing the processes associated with developing and refining data collection tools. Thus, there is little opportunity to share lessons learned and strategies to overcome challenges. In this manuscript, we share our process of developing and iterative refining a data collection tool called SSPOT (Sociotechnical Systems Prospective Observational Tool) with the intended use of conducting observations in clinical settings.

Using SEIPS 2.0 as a guide, we developed and iteratively refined a digital tool (within REDCap) that facilitates capture of observational data in (near) real time across 4 phases: Prototype Development and Refinement, Nonclinical Pilot Testing, Clinical Pilot Testing, and Inter-Observer Consistency.

The resultant tool supports manual documentation of observational data associated with activities, personnel, and tools/technologies. Development took 99 h, including 71 h of observation and 28 h of debriefing over 9 weeks, of which approximately 4 h of pilot testing were in a non-clinical setting. The coffee shop served as a successful alternative sociotechnical system for pilot testing. Once development was completed, the tool was used in an adult and a pediatric ED for three observational studies that cumulatively collected data for 399.08 h.

By sharing the iterative development process, we aim to inform future de novo observational data collection tool development efforts. We recommend that research teams: 1) gather a team with multiple perspectives including human factors 2) plan and budget accordingly (including time) for data collection tool development and refinement, 3) consider leveraging creative and easily-accessible non-clinical settings for testing prior to piloting in the clinical setting.