Surgical applications of the Ultraleap 3Di-based gesture-controlled 3D imaging visualization system

Some laparoscopic surgeries require surgeons to repeatedly refer to the patient's 3D imaging data and/or other medical records during the operation. Consequently, accurate visual information and flexible image retrieval are critical to ensure surgical safety and efficiency. Herein, we proposed a 3D imaging visualization system based on an Ultraleap 3Di stereo hand-tracking camera which could offer surgeons a real-time and intuitive display of anatomical organ features. The system uses 3D reconstruction of organs to accurately visualize lesions, as well as important vessels and nerves, facilitating rapid and effective decision-making in complex surgical environments. During system evaluation, clinical tests were conducted by 10 thoracic surgeons. The performance and gesture recognition success rates of the proposed system under different scenarios were investigated, and the task completion time and satisfaction were collected. The results showed that the Ultraleap 3Di gesture control 3D imaging visualization system showed significantly enhanced operation efficiency of thoracic surgeons and high accuracy of gesture recognition in basic operations and in complicated scenarios. Considering the user-friendly interface and efficient gesture recognition of Ultraleap 3Di, surgeons can easily navigate complex anatomical structures, in which shortens the operation time and complication risk. This study provides insights into the digital transformation of surgical procedures and facilitates the development of surgical visualization and control protocols. Nonetheless, future studies should further assess the proposed system's clinical effectiveness and applicability, contributing to the intelligent and refined transformation of clinical surgery.