A CNN-based prototype method of unstructured surgical state perception and navigation for an endovascular surgery...

Performance of robot-assisted endovascular surgery (ES) remains highly dependent on an individual surgeon's skills, due to common adoption of master-slave robotic structure. Surgeons' skill modeling and unstructured surgical state perception pose prohibitive challenges for an autonomous ES robot. In this paper, a novel convolutional neural network (CNN)-based framework is proposed to address these challenges for navigation of an ES robot based on surgeons' skill learning. An operating action probability estimator is proposed by integrating a two-dimensional CNN, with which the features of a surgical state image are extracted and then directly mapped to the action probability. A one-dimensional CNN with multi-input is developed to recognize the guide wire operating force condition. An eye-hand collaborative servoing algorithm is proposed to combine the outputs of these two networks and to control the robot under a closed-loop architecture. A real-world ES robot is employed for data collection and task performance evaluation in laboratory condition. Compared with the state of the art, the CNN-based method shows its capability of adapting to different situations and achieves similar success rate and average operating time. Robotic operation performs similar operating trajectory and maintains similar level of operating force with manual operation. The CNN-based method can be easily extended to many other surgical robots. Graphical abstract A surgeon's guide wire operating skills in endovascular surgery (ES) is learned by the proposed CNN-based method. Then, the learned model is used for autonomous control of a ES robot with surgical state input (images and operating force).