Using OR Black Box Technology to Determine Quality Improvement Outcomes for In-situ Timeout and Debrief Simulation.

Area of Science:

• Surgical Safety
• Quality Improvement
• Medical Simulation

Background:

• Operating Room Black Box (ORBB) technology enables real-time assessment of surgical safety checklist performance.
• Pre-study baseline data showed suboptimal performance in timeout and debrief quality, compliance, and engagement.
• Existing timeout quality was 73.3%, compliance 99.9%, and engagement 89.7%; debrief quality was 76.0%, compliance 66.9%, and engagement 66.7%.

Purpose of the Study:

• To evaluate quality improvement outcomes after implementing an in-situ simulation program.
• To enhance surgical safety checklist performance through simulation-based training.
• To assess the impact of simulation on operating room team dynamics and safety culture.

Main Methods:

• A prospective, convergent multi-methods study involving a 15-minute in-situ simulation with rapid cycle deliberate practice for operating room teams.
• ORBB analytics were used to collect Timeout and Debrief scores for surgeons participating in simulation (Sim-group) versus those who did not (No-sim group) over a 6-month period.
• Inductive content analysis of simulation discussions was performed to understand team member perspectives.

Main Results:

• Thirty simulations were conducted with 163 interprofessional participants; ORBB data from 1570 cases were analyzed.
• The Sim-group demonstrated significantly improved debrief quality (84% vs. 79%), compliance (73% vs. 66%), and engagement (80% vs. 73%) compared to the No-sim group.
• No significant differences were found in Timeout scores between the groups; thematic analysis revealed themes of "culture of safety" and "policy."

Conclusions:

• Simulation-based quality improvement interventions can foster a psychologically safe training environment for operating room teams.
• The use of ORBB technology effectively facilitated the analysis of simulation outcomes.
• Simulation-trained surgeons showed significantly better debrief scores compared to their non-trained counterparts.