Methodology and bias in assessing compliance with a surgical safety checklist

Surgical safety checklists, such as the perioperative time-out, have been shown to improve performance on a variety of patient safety measures. A variety of methods have been used to assess compliance with the perioperative time-out, but no standardized methodology with a reliable observer group currently exists. An observation-based methodology was used to assess time-out compliance at an academic medical center.

A single observer group made up of medical students and nurses recorded compliance with each of the 11 standardized items of the time-out. A total of 193 time-out procedures were observed, 48 by medical students and 145 by nurses.

One item (procedure to be performed) achieved > 95% compliance. Three items (surgical site; availability of necessary blood products, implants, devices; and start of antibiotics) achieved 80%-95% compliance. Seven items achieved < 80% compliance (presence of required members of procedure team, presence of person who marked patient, patient identity, side marking, relevant images, allergies, and discussion of relevant special considerations). Compliance with the four core time-out items was 78.2%. Of the 11 items on the time-out being evaluated, there was a statistically significant difference between medical student and nursing observations for 10 items (p < .05).

In our cohort of observed time-outs, the compliance rate was low, calling into question time-out quality, and, more importantly, patient safety. Measures must be taken by large hospitals to regularly audit time-out compliance and create effective programming to improve performance. Although observational assessment is an effective method to assess compliance with surgical safety checklists, observer group bias has the potential to skew results.