Video-based detection of device interaction in the operating room

The establishment of modern workflow management technologies requires the integration of dated devices. The extraction of the essential device data and usage time spans is a central requirement for an integrated OR environment. Therefore, methods are required that extract such information from the output provided by older generation devices, namely video signals. We developed a four-level approach for video-based device information extraction. Usually, video streams contain all relevant patient data and device usage information. We propose an approach consisting of defining regions of interest, grabbing video signals, analyzing the signals and storing the data in a centralized and structured location. The analysis considers textual information and graphical visualization. A prototype of the analysis approach was implemented and applied to a neurosurgical case. An evaluation study was conducted to measure the performance of the approach on video recordings of real interventions. Three medical devices were considered: intraoperative ultrasound, neuro-navigation and microscope. Overall, recognition rates for device usage higher than 95% were obtained. The approach is not limited to a single surgical discipline and does not require modification of medical devices. Furthermore, the analysis of microscopic video streams expands the detectable aspects of the surgical workflow beyond the recognition of device usage.