Operating room clinicians' ratings of workload: a vignette simulation study

Increased clinician workload is associated with medical errors and patient harm. The Quality and Workload Assessment Tool (QWAT) measures anticipated (pre-case) and perceived (post-case) clinical workload during actual surgical procedures using ratings of individual and team case difficulty from every operating room (OR) team member. The purpose of this study was to examine the QWAT ratings of OR clinicians who were not present in the OR but who read vignettes compiled from actual case documentation to assess interrater reliability and agreement with ratings made by clinicians involved in the actual cases.

Thirty-six OR clinicians (13 anesthesia providers, 11 surgeons, and 12 nurses) used the QWAT to rate 6 cases varying from easy to moderately difficult based on actual ratings made by clinicians involved with the cases. Cases were presented and rated in random order. Before rating anticipated individual and team difficulty, the raters read prepared clinical vignettes containing case synopses and much of the same written case information that was available to the actual clinicians before the onset of each case. Then, before rating perceived individual and team difficulty, they read part 2 of the vignette consisting of detailed role-specific intraoperative data regarding the anesthetic and surgical course, unusual events, and other relevant contextual factors.

Surgeons had higher interrater reliability on the QWAT than did OR nurses or anesthesia providers. For the anticipated individual and team workload ratings, there were no statistically significant differences between the actual ratings and the ratings obtained from the vignettes. There were differences for the 3 provider types in perceived individual workload for the median difficulty cases and in the perceived team workload for the median and more difficult cases.

The case difficulty items on the QWAT seem to be sufficiently reliable and valid to be used in other studies of anticipated and perceived clinical workload of surgeons. Perhaps because of the limitations of the clinical documentation shown to anesthesia providers and OR nurses in the current vignette study, more evidence needs to be gathered to demonstrate the criterion-related validity of the QWAT difficulty items for assessing the workload of nonsurgeon OR clinicians.