Development of an artificial intelligence based virtual tool for measuring distances during image-guided surgery

Image-guided surgery has unique depth perception challenges. This complicates procedures requiring intracorporeal measurements, including gastric bypass, where conventional methods are subjective. Computer vision (CV) has been used for tool identification, which can locate key features for a mathematics-based prediction of 3D distance. This feasibility study aims to develop such a CV tool to objectively measure intraoperative distances.

Development of the proof-of-concept digital ruler involved developing a CV instrument detection algorithm, and a computer program to compute and display inter-grasper distance. These were then combined and validated. The CV algorithm was trained by annotating laparoscopic surgery videos to identify the jaw assembly. Model performance was tested against ground truth annotations. The computer program was then developed and tested with manual annotations in a bench-box simulator, using a ruler for ground truth. Both components were combined in a prototype for beta-testing and validation in simulation setting, using a bench box and surgery video recordings. Bench box validation compared pipeline and human predictions to actual measured lengths of simulated bowel. Video validation compared pipeline predictions to those shown by an intracorporeal ruler.

A total of 1205 frames (64 cases) were annotated. The model was trained using a 60/20/20 training/testing/validation split. Compared to annotations, the model had a Precision Recall AUC, accuracy, and Dice Score of 0.89, 0.99, and 0.80, respectively. Forty-nine sample measurement frames were used to validate the computer program, with a mean error of estimation of 0.79 cm. Bench box testing compared to a test group showed the prototype's best performance at larger distances (150 cm), with a "human in the loop" system. In the video validation, the prototype demonstrated low measurement variability.

CV-based techniques can be effectively used to reduce subjectivity of intracorporeal measurement by delivering an objective measurement during image-guided surgery.