[Computer-vision-based artificial intelligence for detection and recognition of instruments and organs during radical...

Objective: To investigate the feasibility and accuracy of computer vision-based artificial intelligence technology in detecting and recognizing instruments and organs in the scenario of radical laparoscopic gastrectomy for gastric cancer. Methods: Eight complete laparoscopic distal radical gastrectomy surgery videos were collected from four large tertiary hospitals in China (First Medical Center of Chinese PLA General Hospital [three cases], Liaoning Cancer Hospital [two cases], Liyang Branch of Jiangsu Province People's Hospital [two cases], and Fudan University Shanghai Cancer Center [one case]). PR software was used to extract frames every 5-10 seconds and convert them into image frames. To ensure quality, deduplication was performed manually to remove obvious duplication and blurred image frames. After conversion and deduplication, there were 3369 frame images with a resolution of 1,920×1,080 PPI. LabelMe was used for instance segmentation of the images into the following 23 categories: veins, arteries, sutures, needle holders, ultrasonic knives, suction devices, bleeding, colon, forceps, gallbladder, small gauze, Hem-o-lok, Hem-o-lok appliers, electrocautery hooks, small intestine, hepatogastric ligaments, liver, omentum, pancreas, spleen, surgical staplers, stomach, and trocars. The frame images were randomly allocated to training and validation sets in a 9:1 ratio. The YOLOv8 deep learning framework was used for model training and validation. Precision, recall, average precision (AP), and mean average precision (mAP) were used to evaluate detection and recognition accuracy. Results: The training set contained 3032 frame images comprising 30 895 instance segmentation counts across 23 categories. The validation set contained 337 frame images comprising 3407 instance segmentation counts. The YOLOv8m model was used for training. The loss curve of the training set showed a smooth gradual decrease in loss value as the number of iteration calculations increased. In the training set, the AP values of all 23 categories were above 0.90, with a mAP of 0.99, whereas in the validation set, the mAP of the 23 categories was 0.82. As to individual categories, the AP values for ultrasonic knives, needle holders, forceps, gallbladders, small pieces of gauze, and surgical staplers were 0.96, 0.94, 0.91, 0.91, 0.91, and 0.91, respectively. The model successfully inferred and applied to a 5-minutes video segment of laparoscopic gastroenterostomy suturing. Conclusion: The primary finding of this multicenter study is that computer vision can efficiently, accurately, and in real-time detect organs and instruments in various scenarios of radical laparoscopic gastrectomy for gastric cancer.