Related Experiment Video
Updated: May 29, 2026
Automatic Surgery in Transcatheter Aortic Valve Replacement Using Augmented Reality
Published on: August 9, 2024
Automated recognition of surgical phases in robot-assisted radical prostatectomy using a Hiera-based vision
Liang Liu1,2, Jian-Feng Huang1, Rong-Hui Shi2
1Urology Department, Chongqing University Central Hospital, Fourth People's Hospital of Chongqing, Chongqing Emergency Medical Center, Chongqing University, Chongqing, China.
Objectives:
To construct a reliable robot-assisted radical prostatectomy (RARP) surgical phase-recognition model and sought to deploy the model within the SurgSmart platform (Chengdu Withai Innovations Technology Co., Ltd., Chengdu, China) to explore its feasibility and preliminary clinical utility in real surgical workflows, including its usability, interpretability, and potential value for surgical education and quality review.
Methods:
A total of 72 complete RARP procedure videos were collected, and divided into training, verification and test groups (7:1:2). We adopted Hiera (Meta Platforms, Inc., Menlo Park, CA, USA), a hierarchical vision transformer, as the backbone model for surgical phase recognition. Model performance was evaluated using precision, recall, F1-score, and overall accuracy. The trained model was deployed on the SurgSmart platform and tested in two real world RARP procedures to evaluate its feasibility for intraoperative and postoperative use.
Results:
Temporal annotation quality was high, with a mean inter-annotator Intersection over Union (IoU) score of 0.99 and a weighted IoU score of 0.97. Based on the finalised annotations, the trained Hiera model achieved a weighted F1-score of 0.91, a macro F1-score of 0.90, and an overall accuracy of 0.91 on the test set. Across the seven surgical phases, 'urethral anastomosis' and 'intrafascial dissection' reached F1-scores of 0.96 and 0.92, and the remaining phases demonstrated F1-scores within the 0.85-0.96 range. The confusion matrix demonstrated that most surgical phases were correctly classified, with a high concentration of samples along the diagonal cells, indicating strong alignment between predicted and ground-truth labels. During real-time deployment, the system processed live surgical video streams continuously and generated phase predictions throughout the operation without interruption. All outputs were produced without processing errors or interface interruptions, confirming stable operation in both real-time and retrospective modes.
Conclusion:
Accurate surgical phase recognition might be achievable in RARP under a limited but high-quality data setting, although further validation with larger and more diverse datasets is needed to confirm these findings.
