Intraoperative navigation system for liver resection based on edge-AI and multimodal AI

Traditional intraoperative navigation methods show insufficient adaptability in dynamic surgical environments. The rapid development of Artificial Intelligence (AI) presents an opportunity to overcome these limitations, making the construction of real-time, adaptive intraoperative navigation systems a key research goal. This study, based on Edge-AI and multimodal AI technologies, aims to develop and evaluate a foundational system for achieving real-time, offline intraoperative navigation and warnings during minimally invasive liver surgery.

161 minimally invasive liver resection videos collected from the Medtechshare platform were finely annotated. The dataset was divided into training (60%), validation (20%), and independent test (20%) sets. The self-developed ONE-PEACE-SZYYv3 model was used for training and validation, followed by quantitative evaluation using accuracy, recall, precision, Intersection over Union (IoU), and Dice coefficient on the independent test set. For qualitative evaluation, an innovative Turing-like test method, the Humanlike Test, was proposed. In this test, experienced surgeons blindly evaluated and scored AI-generated warnings and manually delineated warnings created by junior surgeons. Two groups of data from different processing sources were blindly evaluated by experts, combined with Mann-Whitney U and TOST (Δ =  ± 1.5 points) tests to assess whether they were equivalent within an acceptable range. This ultimately tested whether the current AI possesses judgment capabilities plausible enough to be comparable to human levels during surgery.

In quantitative evaluation on the independent test set, under mAP50, the model's mean average precision reached as high as 99.5%, and under the stricter mAP50-95, it achieved 89.9% precision. IoU and Dice were 0.63 and 0.79, respectively. In qualitative evaluation, the average score of the "AI guided results, Experimental group" was slightly lower than the "Manually delineated results, Control group," but the difference was not statistically significant (p > 0.05). Equivalence testing confirmed that the scores were statistically equivalent within a pre-defined narrow margin.

The proposed model exhibits high precision and real-time capability on a curated, retrospective dataset. It can accurately and effectively provide multimodal guidance (image-text-sound) and warnings in complex anatomical structures with multiple types of interference elements during surgery. The innovative Humanlike Test shows its intraoperative judgment capability can, to a certain extent, produce outputs that are indistinguishable from those of human surgeons. While this study demonstrates foundational feasibility, prospective clinical trials are required to validate its clinical utility and impact on surgical outcomes. Such technology holds promise for elevating the surgical field to a new paradigm of digital intelligence.