Multitask Imaging Monitor for Surgical Navigation: Combination of Touchless Interface and Head-Mounted Display

As a result of the dramatic improvements in the resolution, wearability, and weight of head-mounted displays (HMDs), they have become increasingly applied in the medical field as personal imaging monitors. The combined use of a multiplexer with an HMD allows the wearer to simultaneously and seamlessly monitor multiple streams of imaging information through the HMD. We developed a multitask imaging monitor for surgical navigation by combining a touchless surgical imaging control system with an HMD. This system is composed of a standard color digital video camera mounted on the HMD and computer software that enables the identification of the number of pictured fingertips from the video camera image. The HMD wearer uses this information as a touchless interface for the operating multiplexer, which can control the arrays and types of imaging information displayed on the HMD. We used this system in an experimental demonstration during a single-port gasless partial nephrectomy. The use of this multitask imaging monitor using a touchless interface would refine the surgical workflow, especially during surgical navigation.